How to Use Terraform Providers

What are Terraform Providers

A key feature of Terraform is the ability to manage infrastructure on virtually any platform. But how does Terraform know how to interact with infrastructure services as diverse as AWS, Kubernetes, and GitHub? That's where Terraform providers step in. They are the superpower that lets Terraform deploy infrastructure to any cloud or service.

A Terraform provider plugin is an executable binary that implements the Terraform plugin framework. It creates a layer of abstraction between the provider's upstream APIs and the constructs Terraform expects to work with. 

Terraform core doesn't know how the provider APIs work, it only knows how to manage resources and data sources. A Terraform provider is responsible for understanding API interactions and translating exposing resources and data sources from a cloud provider like AWS to a framework Terraform understands.

A Terraform provider encapsulates authentication methods, resources supported, lifecycle management, and API calls. Without a rich provider ecosystem backed by a robust community, Terraform wouldn't be able to provision infrastructure or manage resources. Providers are pretty important!

How to Install Terraform Providers

The public Terraform registry has thousands of published providers, including the most popular cloud providers. Each Terraform provider on the registry is open-source, free to use, and includes documentation for using the provider in your Terraform configurations. When adding a new Terraform provider to your code, you should define its properties in the [.code]required_providers[.code] block and then configure instances of the provider using [.code]provider[.code] blocks.

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.0.0"
    }
  }
}
provider "azurerm" {
  features {}
}

Terraform will attempt to infer which providers are being used based on the resource types in your configuration, but explicitly defining providers gives you more control over the version and source being used.

Once you've added providers to your Terraform configuration file, the next step is to run [.code]terraform init[.code] which downloads and installs providers from the sources listed in the [.code]required_providers[.code] block.

> terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "3.0.0"...
- Installing hashicorp/azurerm v3.0.0...
- Installed hashicorp/azurerm v3.0.0 (signed by HashiCorp)

The Terraform providers are packaged as plugin binaries. They are downloaded and stored in the .terraform/providers directory in your current working directory.

Many providers require configuration before they can be used to manage resources. For instance, the AWS provider needs to know which AWS region you plan to provision resources in, and the Terraform Kubernetes provider needs to know the address of the Kubernetes cluster you'd like to connect to.

Provider configuration is defined in a [.code]provider[.code] block using the provider keyword and the name of the provider from the [.code]required_providers[.code] block. If you need to create more than one instance of a Terraform provider, you can do so using the [.code]alias[.code] argument inside of the [.code]provider[.code] block. For example, you may want to work with multiple regions in AWS:

provider "aws" {
  region = "us-east-1"
}

provider "aws" {
  alias = "west"
  region = "us-west-1"
}

Few Things to Know

With thousands of providers available on the public Terraform registry, you might wonder how to select the right Terraform provider for your use case. Providers on the registry have one of three designations:

1. Official provider – These providers are collaboratively maintained by a dedicated team at HashiCorp, usually with assistance from the service vendor or working group.
2. Partner provider – These providers are maintained by an organization that has completed the HashiCorp Technical Partner requirement.
3. Community provider – These providers are maintained by an individual or group of contributors.

Official and partner providers tend to have the most active community behind them, have a regular release cadence, and respond quickly to issues and requests. You can also view the popularity of a provider based on downloads, the most recent release, and the provider documentation from the provider overview page.

Because providers from the registry are open-source and hosted on GitHub, you can also view the source code for each provider, browse through its current issues, and read the release notes for each version to see whether to upgrade.

Providers on the public registry are versioned using the semantic versioning standard of Major.Minor.Patch. You can constrain the version being used by your configuration using the [.code]version[.code] argument in the [.code]required_providers[.code] block. 

Newer versions of a Terraform provider will include new resources, additional arguments for existing resources, and bug fixes for issues. Major version updates may also include deprecated resource types, arguments, or other breaking changes.

Controlling the provider version gives you and your team a consistent experience when working on Terraform code. To make the version upgrade process more deliberate, HashiCorp introduced the Terraform dependency lock file.

The dependency lock file is created the first time [.code]terraform init[.code] is run against a Terraform configuration. It captures the current version of each Terraform provider being used and creates the file .terraform.lock.hcl in the configuration directory to capture that information.

provider "registry.terraform.io/hashicorp/azurerm" {
  version     = "3.77.0"
  constraints = "~> 3.0"
  hashes = [
    "h1:7XzdPKIJoKazb3eMhQbjcVWRRu5RSsHeDNYUQW6zSAc=",
    "zh:071c82025cda506af90302b3e89f61e086ad9e3b97b8c55382d5aed6f207cf10",
    ...
  ]
}

Also included in the file is the constraint for the provider version and a hash of the provider plugin executable.

Although all Terraform providers use the same Terraform plugin SDK to expose infrastructure resources and data sources from their upstream API, different providers have their own arguments and supported authentication methods. Let's review some of the most popular providers on the Terraform registry.

What are the most popular Terraform providers?

The most commonly used providers are listed on the main provider page of the Terraform public registry:

The following table shows the six most popular providers, their total downloads, and current version:

Each provider comes with documentation detailing the infrastructure resources and data sources contained within and how to configure and interact with the provider. Of particular note is the way in which each provider handles authentication to their service.

The three major cloud providers – AWS, Azure, and Google – each have multiple ways to authenticate depending on the client type and usage scenario. 

For instance, the AWS provider supports authentication from environment variables, instance profiles, container credentials, and shared credential files. Before attempting to integrate a new provider into your configuration, be sure to read the documentation, especially the authentication section.

Some providers also handle preview, beta features, and services differently. 

The Azure provider does not immediately support all functionality exposed by the Azure Resource Manager API (in particular for features and services that are in private or public preview). As a stopgap measure, the AzAPI provider is available, which is a thin layer of abstraction over the ARM API. 

Likewise, the Google Cloud Platform provider does not support beta features and services on the platform. If you wish to use a beta feature, you can leverage the Google Beta instead.

Let's take a closer look at the most popular provider on the registry, the AWS provider.

Example: Terraform AWS Provider

Amazon Web Services (AWS) was the first external provider included in Terraform. It remains the most popular provider based on download metrics by a comfortable margin. Each instance of the AWS provider in a configuration is region-specific, and the region is set using either an argument in the provider configuration block or with the [.code]AWS_REGION[.code] environment variable.

provider "aws" {
  region = "us-west-2"
}

When adding an AWS provider to your Terraform code, you need to determine how you'll authenticate to the platform. The most common method when running from a workstation is to use the Access Key and Secret Key stored in your AWS credentials file. Typically, this is generated using the AWS CLI command [.code]configure[.code].

> aws configure
AWS Access Key ID: *********************
AWS Secret Access Key: ***********************
Default region name: us-west-2
Default output format: json

The [.code]configure[.code] command will create or update the default profile stored in your credentials file. The AWS provider will use the default profile if no other authentication methods are configured.

You can work with multiple AWS accounts in your configuration by adding additional provider configuration blocks with the [.code]alias[.code] meta-argument. If you have a cross-account role to assume, that can be configured use the [.code]assume_role[.code] argument.

provider "aws" {
  alias       = "shared"
  assume_role = "shared-ec2-admin"
}

Final Thoughts

Terraform providers are essential to the functionality of Terraform. Without provider plugins, Terraform wouldn't be able to provision and manage infrastructure! A large part of Terraform's success is owed to the vast collection of Official, Partner, and Community providers backed by vibrant and active groups of maintainers.

Terraform providers published on the public registry are all open-source and hosted on GitHub. Because the provider schema is well-documented, Terraform providers are not necessarily limited to only being consumed by Terraform.

OpenTofu, an open-source alternative to Terraform, works perfectly with the existing Terraform providers. Pulumi, an infrastructure management tool leveraging general-purpose programming languages, also uses Terraform providers when a native provider for the platform is not available.