Terraform State File Management (Local and Remote)

In this article, we are going to cover the details of managing state files in Terraform | Terraform state file Management providing insights into optimal practices for robust infrastructure deployment.

Managing Terraform State Files 1 2

What is Terraform State File?

The Terraform state file, typically named “terraform.tfstate,” is a crucial artifact that stores the current state of deployed infrastructure. It contains a snapshot of resource attributes and their relationships as defined in the Terraform configuration. This file enables Terraform to track changes, synchronize with the actual infrastructure, and facilitate accurate updates or modifications when the configuration evolves. Proper management of the state file is essential for maintaining consistency and ensuring effective collaboration in infrastructure as code projects.

What is the purpose of the State File in Terraform?

The Terraform state file maintains the current state of managed infrastructure, enabling accurate tracking, preventing concurrent conflicts, and facilitating recovery and rollbacks in case of errors. It also stores resource metadata and output values for effective resource management.

Key Purpose of State File in Terraform:

  1. State Tracking: Keeps track of the current state of managed infrastructure resources.
  2. Concurrency Control: Prevents conflicts by locking the state during concurrent operations.
  3. Resource Metadata: Stores metadata for each resource, aiding in accurate management and updates.
  4. Output Values: Records the current values of defined outputs for querying post-deployment.
  5. Recovery and Rollbacks: Facilitates recovery from errors and rollbacks by storing previous infrastructure states.

How does Terraform State Work?

Terraform state works by storing a record of the resources deployed and managed by Terraform and their current configuration. When you run the terraform plan and terraform apply commands, it uses the contents of the state file to determine the required actions to modify the infrastructure resources to bring them into the desired state that is configured in the Terraform code. The state is also used to determine if a resource needs to be created, updated, or destroyed in order to meet the desired configuration.

Terraform state can be managed locally or remotely. Local state management stores the state file on the local file system, while remote state management stores the state file in a remote data store such as Microsoft Azure Blob Storage or AWS S3. Remote state management provides better collaboration and security features than local state management.

Terraform state management can be divided into two categories:

  • Terraform Local state management
  • Terraform Remote state management

Difference Between Local And Remote State Management

AspectLocal State ManagementRemote State Management
Storage LocationStored on the machine where Terraform is executed.Stored remotely in a shared backend (e.g., S3, Azure Storage).
Concurrency ControlNo built-in concurrency control; prone to conflicts.Built-in concurrency control to prevent conflicts during operations.
CollaborationLimited collaboration; challenging for team use.Facilitates collaboration by providing a central, shared state.
SecurityPotentially less secure, as the state is on a local machine.Improved security as state is stored in a secure, centralized location.
Backup and RecoveryRelies on manual backups and is susceptible to data loss.Easier backup and recovery options, with versioning and history tracking.
Access ControlLimited access control; relies on file permissions.Enhanced access control through backend authentication and authorization.
Terraform Commandsterraform apply and terraform destroy may lead to local state inconsistencies.Safely apply changes using terraform apply without risking local inconsistencies.
ScalingMore challenging to scale for large teams and complex infrastructures.Scales well for teams and larger infrastructures due to centralized state management.
Offline WorkSuitable for offline work without requiring remote access.Requires network access to the remote backend, limiting offline capabilities.
Difference Between Terraform Local And Terraform Remote State Management

Terraform Local State Management

Local state management is the default approach employed by Terraform. Under this method, Terraform creates a local state file in the same directory as your Terraform configuration files to facilitate state administration locally. Here’s how you can accomplish it:

Initializing a Terraform Project

The first step is to initialize your Terraform project. This command sets up the necessary plugins and backend configurations.

terraform init
Screenshot 2023 11 16 145217 1

Creating Resources

Define infrastructure resources in your Terraform configuration file (e.g., main.tf). For example, to launch an AWS EC2 instance:

resource "aws_instance" "myec2" {
    ami = "ami-0cbd40f694b804622"
    instance_type = "t2.micro"

}
Screenshot 2023 11 16 150326

Apply the configuration to create the resources:

terraform apply 
Screenshot 2023 11 16 150111
Screenshot 2023 11 16 150126

Terraform will add the details of the provisioned resource in the terraform.tfstate file. 

Modifying Resources

To modify existing resources, make changes to your Terraform configuration and apply the changes again. For instance, we can add a tag to the resource created in the previous step.

provider "aws" {
 profile = "default"
}
resource "aws_instance" "myec2" {
    ami = "ami-0cbd40f694b804622"
    instance_type = "t2.micro"

    tags = {
        Name = "server2"
    }
  
}
Screenshot 2023 11 16 143930 1

Apply the configuration to modify the resources:

terraform apply

Terraform will identify any discrepancies between your desired state in your configuration and what is currently stored in the local state file and make any necessary updates.

Deleting Resources and Cleanup

To delete resources, remove them from your configuration file and apply the changes.

Screenshot 2023 11 16 150631
Screenshot 2023 11 16 150658

The resources will be destroyed, and the state file will be updated to reflect the changes.

Challenges with Terraform Local State Management

image 7

Below are the limitations of local state management:

  • Limited Collaboration: Local state files are tied to individual machines, making it more difficult for teams to collaborate effectively.
  • Risk of Data Loss: Local state files are vulnerable to data loss if a machine crashes or the state file is accidentally deleted.
  • Concurrency Issues: Within teams, multiple members may attempt to implement changes simultaneously, leading to conflicts in configuration.

Terraform Remote State Management

image 11

Remote state management is a more robust approach suitable for team environments and production use cases. In this method, the Terraform state file is stored remotely in a shared location that all team members can access. Common choices for remote state storage include Amazon S3, Azure Blob Storage, and Terraform Cloud.

Here’s how to set up remote state management:

Initialize Remote State

Initialize your Terraform project:

terraform init

However, instead of using the default local state backend, specify a remote state backend in your configuration.

Configure Terraform Remote State Backend

In your Terraform configuration (e.g., main.tf), specify the backend configuration to use the remote state. For example, using Amazon S3 as the remote state backend: 

Screenshot 2023 11 17 162619
provider "aws" {
 profile = "default"
}
resource "aws_instance" "ec2_instance" {
    ami = "ami-0f98e53cd29272444"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }

  
}
terraform {
    backend "s3" {
        bucket = "my-aws-storagebucket-1"
        key  = "S3/terraform.tfstate"
        region     = "us-west-1"
        shared_credentials_file = "~/.aws/credentials"
        
    }
}

In this example, we use an S3 bucket to store the state file and DynamoDB for locking. Below are the details of the above configurations:

  • bucket: This is the name of the S3 bucket where Terraform will store its state file and related data. 
  • key: This is the path within the S3 bucket where Terraform will store its state file. 
  • region: This specifies the AWS region in which the S3 bucket is located.
  • encrypt: When set to true, it means that Terraform will encrypt the state file when storing it in S3, providing additional security.

Apply Changes with Remote State

Apply your Terraform configuration using the command below:

terraform apply

Terraform will store the state remotely in the specified backend, making it accessible to all team members.

image 12

Benefits of Terraform Remote State Management:-

Remote state management offers several advantages:

  • Improved collaboration: All team members can now access and update the same state file without manually sharing and synchronizing state files.
  • Increased security: Remote state storage solutions often have built-in security features, including access control mechanisms.
  • Better data protection: Cloud-based remote state solutions typically offer data redundancy, backups, and versioning to protect state data from accidental deletion or corruption.

We have covered Terraform State file(Local and Remote).

Conclusion:-

Effectively managing state files in Terraform is crucial for maintaining infrastructure integrity, ensuring collaboration, and enabling seamless updates, ultimately enhancing the efficiency of infrastructure as code workflows.

Reference:-

For reference visit the official website TerraformRegistry.

Any queries pls contact us @Devopshint.

Related Articles:

HashiCorp- DevOps Infrastructure Provisioning and Management Products

About Akash Bhujbal

Hey, I am Akash Bhujbal, I am an aspiring DevOps and Cloud enthusiast who is eager to embark on a journey into the world of DevOps and Cloud. With a strong passion for technology and a keen interest in DevOps and Cloud based solutions, I am driven to learn and contribute to the ever-evolving field of DevOps and Cloud.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Share via
Copy link