Terraform: a brief introductionPublished on: Author: Nik Zhigalov Category: IT development and operations
Infrastructure is not easy to manage. It is, at times, difficult to set up and even more difficult to maintain as your application evolves to meet your client’s needs. Thanks to a shift in philosophy in the last decade and a lot of new tools, there has been a massive rise in the DevOps field. The goal is to speed up the ability of delivering services by merging the fields of software development and infrastructure operations. One of the newer tools is Terraform, whose capabilities we will briefly explore in this article to see how it can help us in our development lifecycle.
Let’s start with a scenario that is probably familiar to many of you. You come to work one day and realize that, for some reason, you have to migrate your application to different machines or to a different cloud provider.
This means you have to go over any potential documentation that is likely out of date, on portals such as Confluence. You also need to have numerous discussions with your teammates and spend days if not weeks recreating the environments.
On top of that, when you’re finished you may likely discover that the application no longer functions as intended due to configuration drift - meaning server setups have slight differences between them in terms of minor version differences.
Terraform to the rescue
This is the type of situation where Terraform, in conjunction with other tools, can help you. So, what is Terraform? Simply put, Terraform is a declarative architecture orchestration tool that allows you to provision, configure, and evolve your immutable architecture. Sounds great, but what does this really mean? And how does it actually benefit you?
Orchestrate the infrastructure you need
The first key point is that we can orchestrate the infrastructure we need, meaning we specify in code what servers we need and Terraform makes the required API calls to the provider, to make sure the servers exist in the state you specified.
Other tools, such as Ansible and Chef are meant to configure already existing servers, not provide you with new instances. On top of this you get the advantage of having your architecture defined in code. Here you can commit it to a repository, perform code review before any changes are made, and have an auditable trace of all the changes ever made.
Another key benefit of using Terraform is that in the configuration files, denoted by the .tf extension, we declare what we need and how we need it without caring how we get there. Take a look at this AWS instance example:
Here, we declared that we need five t2.micro instances named hello_world in west Europe. Adding some credentials and running terraform apply results in five instances being created, and that’s it. You don’t need to know how to navigate the AWS configuration menus.
In addition, Terraform remembers the state of your infrastructure. So if you suddenly decide you need ten instances, change the instance_count value and re-apply the script. Other tools like Chef are procedural, so you would need to rewrite their script to identify the number of running instances and then create the difference.
The third benefit mentioned in the brief summary of Terraform, is that we use it to create immutable infrastructure—or infrastructure whose configuration doesn’t change after setup. While this is not a wholly unique benefit, it is easier to implement here because any infrastructure defined by Terraform is by nature immutable—since you are not defining how you want to get to your end state.
Server immutability is beneficial because you can be sure that all your environments are truly identical, and you won’t have to deal with bugs resulting from software versions mismatching.
If you need to upgrade Java, or anything else for that matter, deploy a new server, verify that it works, and destroy the existing one. Using containerization tools like Docker makes this easier since you will be defining the required software state in those Dockerfiles.
How can we use it?
Coming back to the scenario we started with, let’s make the ideal assumption that our application is running out of Docker containers, is decently integration tested, and is deployed via a Jenkins server. Now we can make the migration easier by integrating Terraform into our stack.
How can we integrate Terraform into our stack?
There are two approaches. The first is to create an environment by manually running our Terraform scripts, then configuring Jenkins to deploy our application(s) to those servers. This would be done by installing Terraform locally, writing up the needed .tf files, and creating the defined architecture via the terraform apply command.
Alternatively, we could integrate our Terraform scripts into our application by adding the Terraform files into a dedicated module, installing Terraform on our Jenkins server, and finally having our Jenkins pipeline deploy the infrastructure.
This way we can create tests with tools such as InSpec or Goss. As usual, there is an overhead cost to TDD, but the reward is that we can be certain our infrastructure looks and behaves just as we intended.
By adding it to the Jenkins pipeline, we can get rid of our testing environment entirely. Use the same Terraform script where you defined your production environment to spin up a new environment; just to run your integration tests off. If they all pass, then your test environment will become your acceptance environment and you can delete your existing acceptance environment since it is now obsolete. When the client accepts the changes, it will become the production environment.
As Carlos Nunez mentioned in his article on Terraform testing: “Every sandbox environment created by an integration test will be an exact replica of production because every sandbox environment will ultimately become production.”
Of course, if you have quite a large infrastructure running your application, deploying a duplicate for test validation can get quite expensive. You could consider deploying a scaled down version instead, and then scaling it up when you’re ready to make it the production environment.
What else can it do?
Terraform isn’t just limited to creating server instances, you can also use it for the following tasks:
- Create clusters
- Set up security policies
- Configure multi-tier applications
- Define load balancers
- Multi-cloud deployment
- Manage software defined networks
- Manage git repositories
- Configure your database
Better than the alternatives?
It is hard to tell. It really depends on your use case, and what issues you and your team are willing to live with. In the articles I have read, there seems to be one standing consensus. A lot of reviews about the alternatives to Terraform seem to cover only the high level benefits. They are unclear about which alternative you should use when—leaving their flaws as a surprise for when you actually start working with them.
What's the catch?
We listed a lot of benefits that make Terraform a great tool, but it’s not a golden goose. It will not magically solve all your infrastructure issues. It is a tool that, when integrated with other tooling in your stack, can help make it easier for you to manage your infrastructure. Let’s look at some weaknesses:
- It’s young: Seeing how Terraform was created in 2014, it’s not quite as mature as some of the other alternatives such as Puppet, which came out in 2005. It is still a work in progress; at the time of this article the latest version is 0.11.7 and there are 1286 open issues on its github page.
- State storing isn’t perfect: Despite the fact that storing infrastructure state is one of Terraforms biggest benefits, it’s not perfect, and there are discussions about issues with this stored state.
- Not exactly provider idempotent: While you can use Terraform with a lot of providers, your one Terraform script won’t work with all of them. This is because all the listed providers offer different services, features, and different configuration requirements.
- Not all provider complexity is hidden: You don’t have to learn to navigate all the providers UI screens, not even when they get redesigned, but you still need to have a good understanding how their services work and how they need to be configured. This is definitely not a trivial matter.
- Not always up to date: If one of the service providers updates the services they offer, you will have to wait for Terraform to be updated.
Final piece of advice: acknowledge the challenges
We can conclude that Terraform is a great tool that can help make your life easier, but that’s not the full story. There are definitely challenges and decisions you have to make before you dive into the world of DevOps.
Want to learn more?
If you found this introduction article interesting and would like to learn more, comment on this post or reach out to me at firstname.lastname@example.org. We could turn this article into a series, where we delve into and learn how to actually use and/or configure Terraform.
Tip: For those of you that prefer to do their own research, I highly recommend the numerous articles written by Yevgeniy Brikman. He even wrote a book on Terraform.