Managing Terraform state effectively is crucial for maintaining the integrity of your infrastructure. As your organization grows, you may need to move Terraform state files between different AWS accounts for security, compliance, or organizational restructuring reasons. This article provides steps how to safely migrate your Terraform state from one AWS account to another.

Why Move Terraform State Between Accounts?

Moving Terraform state files between AWS accounts might be necessary for several reasons:

1. Security: Separating state files from the infrastructure they manage can enhance security by limiting access to sensitive information.
2. Compliance: Regulatory requirements might mandate that state files be stored in specific accounts or regions.
3. Organizational Changes: As your organization evolves, you may need to reorganize cloud resources and associated state files into different accounts.

Pre-Migration Considerations

Before you begin the migration process, ensure the following:

• Backup Your State File: Always back up your Terraform state file before making any changes. This can be done by copying the file to a secure location.
• Plan for Downtime: If your infrastructure is highly sensitive, plan for potential downtime during the migration process.
• Access Permissions: Ensure that you have the necessary permissions in both the source and target AWS accounts to access and modify the Terraform state.

Step-by-Step Guide to Moving Terraform State

1. Set Up the S3 Bucket in the Target Account

First, create an S3 bucket and a DynamoDB table in the target AWS account where you’ll store the Terraform state and manage state locking.

provider "aws" {
  alias  = "target"
  region = "us-west-2"
  # Configure credentials for the target account
}

resource "aws_s3_bucket" "terraform_state" {
  provider = aws.target
  bucket   = "my-new-terraform-state-bucket"

  versioning {
    enabled = true
  }

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }
}

resource "aws_dynamodb_table" "terraform_locks" {
  provider  = aws.target
  name      = "terraform-state-locks"
  hash_key  = "LockID"
  billing_mode = "PAY_PER_REQUEST"

  attribute {
    name = "LockID"
    type = "S"
  }
}

2. Update the Backend Configuration

In your existing Terraform configuration, update the backend block to point to the new S3 bucket and DynamoDB table in the target account.

terraform {
  backend "s3" {
    bucket         = "my-new-terraform-state-bucket"
    key            = "path/to/my/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-state-locks"
  }
}

3. Migrate the State File

To move the Terraform state file to the new S3 bucket, use the terraform init command with the -migrate-state flag. This will prompt Terraform to migrate the state from the current backend to the new one.

terraform init -migrate-state

4. Verify the Migration

After the migration is complete, verify that the state has been correctly moved by running terraform plan. This ensures that Terraform can read the state from the new backend and that there are no discrepancies.

terraform plan

If everything is configured correctly, Terraform should not show any differences in the resources managed by the state file.

5. Clean Up the Old State

Once you have verified that the migration was successful, clean up the old state by deleting the state file from the previous S3 bucket in the original account. This is an optional but recommended step to prevent confusion and ensure that no one accidentally uses the old state.

aws s3 rm s3://my-old-terraform-state-bucket/path/to/my/terraform.tfstate

Post-Migration Considerations

• Update CI/CD Pipelines: If you use continuous integration/continuous deployment (CI/CD) pipelines, update them to reference the new state location.
• Permissions: Review and update permissions in the new AWS account to ensure only authorized users have access to the Terraform state files.
• Monitoring and Logging: Set up monitoring and logging in the target account to track access and changes to the Terraform state.

Troubleshooting Common Issues

• Access Denied Errors: Ensure that the IAM roles or users in the target account have the necessary permissions to access the S3 bucket and DynamoDB table.
• State File Corruption: If the state file appears to be corrupted after migration, restore from your backup and retry the migration process.
• State Locking Issues: If you encounter issues with state locking, verify that the DynamoDB table is correctly configured and that the lock ID matches between the old and new configurations.

Conclusion

Moving Terraform state between AWS accounts is a straightforward process that can significantly improve your cloud infrastructure’s security and compliance posture. By following the steps outlined in this guide, you can ensure a smooth and safe transition of your Terraform state files. Always remember to back up your state files before making any changes, verify the migration process, and update all related configurations accordingly.