Terraform: a brief introductionPublished on: Author: Nik Zhigalov Category: IT development and operations
Infrastructure is not easy to manage. It is, at times, difficult to setup and even more difficult to maintain as your application evolves to meet your clients needs. Thanks to a lot of new tools and a shift in philosophy in the last decade. There has been a massive rise in the DevOps field where the goal is to speed up the ability of delivering services by merging the fields of software development and infrastructure operations. One of the newer such tools is Terraform, whose capabilities we will briefly explore in this article and see how it can help us in our development lifecycle.
Let’s start off with a scenario that likely is familiar to many of you, you come to work one day and realize that for one reason or another you have to migrate your application to different machines or to a different cloud provider.
This will result in you having to go over any potential documentation, that is likely out of date on portals such as Confluence, have numerous discussions with your teammates, and spend days if not weeks recreating the environments.
On top of that, when you’re finished you may likely discover that the application no longer functions as intended due to configuration drift - where server setups have slight differences between them in terms of minor version differences.
Terraform to the rescue
This is the type of situation where Terraform in conjunction with other tools can help you. So, what is Terraform? Simply put, Terraform is a declarative architecture orchestration tool that allows you to provision, configure, and evolve your immutable architecture. Sounds great, but what does this really mean? And how does it really benefit you?
Orchestrate the infrastructure you need
The first key point here is that we can orchestrate the infrastructure we need, meaning we specify in code what servers we need and Terraform makes the required API calls against the provider, to make sure the servers exist in the state you specified.
Other tools, such as Ansible and Chef are meant to configure already existing servers, not to provide you with new instances. On top of this you get the advantage of having your architecture defined in code. Here you can commit it to a repository, perform code review before any changes are made and have an auditable trace of all the changes ever made.
Another key benefit to using Terraform is that in the configuration files, denoted by the .tf extension, we declare what we need and how we need it without caring how we get there. Take a look at this AWS instance example:
Here we declare that we need need five t2.micro instances named hello_world in west Europe. Adding some credentials and running terraform apply results in five instances being created, that’s it. You don’t need to know how to navigate the AWS configuration menus or anything.
In addition to this, Terraform will remember the state of your infrastructure, so if you suddenly decide you need ten instances then change the instance_count value and re-apply the script. Other tools like Chef are procedural, so you would need to rewrite their script to identify the number of running instances then create the difference.
The third thing we mentioned in the brief summary of Terraform is that we use it to create immutable infrastructure - or infrastructure whose configuration doesn’t change after setup. While this is not a fully unique benefit, it is easier to implement here because any infrastructure defined with Terraform is by nature immutable - since you are not defining how you want to get to your end state.
Server immutability is beneficial in the aspect that you can be sure that all your environments are truly identical and you won’t have to deal with bugs resulting from software version mismatching.
If you need to upgrade Java, or anything else for that matter, then deploy a new server, verify that it works and destroy the existing one. Using containerization tools like Docker makes this simpler since you will be defining the required software state in those Dockerfiles.
How can we use it?
Coming back to the scenario we started with, let’s make an ideal assumption that our application is running out of Docker containers, is decently integration tested, and is deployed via a Jenkins server. Now we can make the migration easier by integrating Terraform into our stack
How can we integrate Terraform into our stack?
There are two approaches, first being to create an environment by manually running our Terraform scripts then configuring Jenkins to deploy our application(s) to those servers. This would be done by installing Terraform locally, writing up the needed .tf files and creating the defined architecture via the terraform apply command.
Alternatively we could integrate our Terraform scripts into our application, by adding the Terraform files into a dedicated module, installing Terraform on our Jenkins server, and finally having our Jenkins pipeline deploy the infrastructure.
This way we can create tests for it, with tools such as InSpec or Goss. As usual, there is an overhead cost to TDD. The reward here is that we can be certain that our infrastructure looks and behaves just as we intended.
In addition, by adding it to the Jenkins pipeline we can get rid of our testing environment entirely. Here you use the same Terraform script where you defined your production environment to spin up a new environment just to run your integration tests off of. If they all pass then your test environment now becomes your acceptance environment and you can destroy your existing acceptance environment since it is now obsolete. Then when the client accepts the changes, it becomes the production environment.
As Carlos Nunez mentioned in his article on Terraform testing: “Every sandbox environment created by an integration test will be an exact replica of production because every sandbox environment will ultimately become production.”
Of course, if you have quite a large infrastructure running your application deploying a duplicate for test validation can get quite expensive, but you can consider deploying a scaled down version and then scaling it up when you’re ready to make it the production environment.
What else can it do?
Terraform isn’t just limited to creating server instances, you can also use it to do some of the following tasks:
- Create clusters
- Setup security policies
- Configure multi-tier applications
- Define load balancers
- Multi-cloud deployment
- Manage software defined networks
- Manage git repositories
- Configure your database
Better than the alternatives?
It is hard to tell. It really depends on your use case and what issues you and your team are willing to live with. In the reading I have done for this article there seems to be one standing consensus. A lot of the reviews of the alternatives to Terraform seem to cover only the high level benefits and not make it clear which one you should use when - leaving the negatives of their use as a surprise for when you actually start working with them.
What is the catch?
We have listed a lot of benefits that make Terraform a great tool, but it is not a golden goose, it will not magically solve all your infrastructure issues. It is a tool that, when integrated with other tooling in your stack, can help make it easier for you to manage your infrastructure. Here are some weaknesses:
- It’s young: Seeing how Terraform was created in 2014 it is not quite as mature as some of the other alternatives such as Puppet that came out in 2005. It is still a work in progress; at the time of this article the latest version is 0.11.7 and there are 1286 open issues on its github page.
- State storing isn’t perfect: Despite that storing infrastructure state is one of Terraforms biggest benefits it’s not perfect and there are discussions about issues with this stored state.
- Not exactly provider idempotent: While you can apply Terraform with a lot of providers, your one Terraform script won’t work on all of them. This is because all these listed providers offer different services, features and different configuration requirements.
- Not all provider complexity is hidden: You may not have to learn to navigate all the providers UI screens, or a second time when they get redesigned, but you will still need to have a good understanding how their services work and how they need to be configured. This is definitely a non-trivial matter.
- Not always up to date: If one of the service providers updates the services they offer, then you will have to wait for Terraform to be updated.
Final advise: acknowledge the challenges
So we can conclude Terraform is a great tool that can help make your life easier, but that is not the full story, there are definitely challenges and decisions you have to make before you dive into the world of DevOps.
Want to learn more?
If you found this introduction article interesting and would like to learn more, comment on this post or reach out to me at firstname.lastname@example.org. So we can turn this article into a series where we delve into and learn how to actually use or configure Terraform.
Tip: Those of you that prefer to do your own research, I highly recommend the numerous articles written by Yevgeniy Brikman. He has even written a book on Terraform.