NIXKNIGHT logo

Terragrunt - Moving from Terraform to OpenTofu


Published: Author:

OpenTofu forked from Terraform after HashiCorp's license change and maintains compatibility while staying open source under the MPL-2.0 license.


Why Migrate to OpenTofu?

My main reason for moving to OpenTofu is its out-of-the-box state encryption support. OpenTofu offers advanced state encryption features that, in my opinion, provide better security for sensitive infrastructure data (such as secrets). You can now safely store secrets in your state without worrying about what will happen if some unauthorized person managed to get their hands on the state file. This also helps with meeting compliance requirements for data protection. But these benefits all depend on whether you do everything right.


Setting Up Terragrunt with OpenTofu

In the current state of my repository, I do not think I need to do anything else other than just replacing Terraform binary with OpenTofu. I expect Terragrunt to pick it up automatically.

As of this blog post, the current version of Terragrunt is 0.77.22 and the current version of OpenTofu is 1.9.1. Last time, I used tfenv and tgenv to install Terraform and Terragrunt. I'll use tgenv to update Terragrunt and install tofuenv to install OpenTofu.

Install tofuenv:

git clone https://github.com/tofuutils/tofuenv.git ~/.local/tofuenv

Update ~/.bashrc to remove tfenv and add tofuenv:

private_bin_dirs=(
  "$HOME/.local/tofuenv/bin"
  "$HOME/.local/tgenv/bin"
)

Read ~/.bashrc again:

source ~/.bashrc

Then install/update:

tgenv install 0.77.22
tgenv use 0.77.22
tofuenv install 1.9.1
tofuenv use 1.9.1

Verify State Compatibility with OpenTofu

While we are at it let's ensure that we clean all previous Terragrunt cache. I will add the following function to my ~./bashrc:

...
...
function clean-terragrunt() {
  find . -type f -name '.terraform.lock.hcl' -prune -exec echo -e "Removing {}" \; -exec rm -rf '{}' +
  find . -type d -name '.terragrunt-cache' -exec echo -e "Removing {}" \; -exec rm -rf '{}' +
}

Read the bashrc again:

source ~/.bashrc

Let's see if Terragrunt/OpenTofu complain about anything:

clean-terragrunt
pushd single-account/environments
terragrunt run-all init

Great! OpenTofu hasn’t raised any concerns so far. I saw that initial warning from Terragrunt. We will come back to it later.


Changes to Existing Configuration

We will update our configuration to remove KMS and add OpenTofu state encryption.

Remove KMS Key

Although OpenTofu's native state encryption supports using AWS KMS, I don't intend to use it. I only want passphrase encryption right now to keep things simple. We will remove the dependency block in single-account/environments/global/state_bucket/terragrunt.hcl and update inputs to remove server_side_encryption_configuration that uses AWS KMS:

include {
  path = find_in_parent_folders()
}

include "s3_bucket" {
  path   = "../../_common/s3_bucket.hcl"
  expose = true
}

locals {
  module_vars = include.s3_bucket.locals.env_vars.module_config.state_bucket
}

inputs = merge(
  local.module_vars,
  {
    attach_policy = true
    policy        = jsonencode(
      yamldecode(
        templatefile(
          "${get_repo_root()}/templates/policies/state_bucket_policy.yaml.tftpl",
          {
            bucket_name = local.module_vars.bucket
            account_id  = "${get_aws_account_id()}"
          }
        )
      )
    )
  }
)

Let's remove inputs from single-account/environments/global/state_bucket_kms_key/terragrunt.hcl so that the file looks like this:

include {
  path = find_in_parent_folders()
}

include "kms" {
  path   = "../../_common/kms.hcl"
  expose = true
}

And then run the plan:

terragrunt run-all plan

This will remove the key alias and the S3 server-side encryption configuration. Let's apply this plan:

terragrunt run-all apply -auto-approve

The key is still there:

terragrunt run-all destroy -auto-approve --terragrunt-working-dir global/state_bucket_kms_key

We will keep single-account/environments/_common/kms.hcl in case we need to use KMS elsewhere.

OpenTofu Encryption

Terragrunt documentation mentions an encryption block that generates OpenTofu state encryption code. However, it is very briefly documented and it looks like it doesn't provide any provisions for migrating existing states. Fortunately, it shows what will be generated as a result. We can temporarily combine that by using Terragrunt generate block in single-account/environments/terragrunt.hcl with OpenTofu documentation section that mentions how to deal with an existing project. So our updated file looks like this:

locals {
  common_vars       = yamldecode(file("common.yaml"))
  state_bucket_name = yamldecode(file("global/environment.yaml")).module_config.state_bucket.bucket
  dynamodb_table    = yamldecode(file("global/environment.yaml")).module_config.state_locking_table.name
  state_encryption_passphrase = get_env("TERRAGRUNT_STATE_ENCRYPTION_PASSPHRASE")
}

# Configure Terragrunt to store state files in an S3 bucket
remote_state {
  backend = "s3"
  config = {
    encrypt        = true
    bucket         = local.state_bucket_name
    key            = "${path_relative_to_include()}/state.json"
    region         = local.common_vars.aws.region
    dynamodb_table = local.dynamodb_table
  }

  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
}

# Setup provider
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite"
  contents  = <<EOF
variable "aws_provider_default_tags" {
  type = map
}

provider "aws" {
  region = "${local.common_vars.aws.region}"
   default_tags {
    tags = var.aws_provider_default_tags
  }
}
EOF
}

# Setup encryption temporarily for migrating existing state
generate "encryption" {
  path      = "encryption.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  encryption {
    method "unencrypted" "migrate" {}
    key_provider "pbkdf2" "default" {
      passphrase = "${local.state_encryption_passphrase}"
    }
    method "aes_gcm" "default" {
      keys = key_provider.pbkdf2.default
    }
    state {
      method = method.aes_gcm.default
      fallback {
        method = method.unencrypted.migrate
      }
    }
  }
}
EOF
}

inputs = {
  aws_provider_default_tags = local.common_vars.tags
}

In my previous experience with OpenTofu state encryption, I have noticed that the migration doesn't really work unless you have a change in your plan that will update the state files. Let's add a tag called TF-Distribution in single-account/environments/common.yaml so that we have a change that updates the state files:

terraform:
  remote_modules:
    s3:
      source: git::git@github.com:terraform-aws-modules/terraform-aws-s3-bucket.git///
      version: v3.14.1
    kms:
      source: git::git@github.com:terraform-aws-modules/terraform-aws-kms.git///
      version: v1.5.0
    dynamodb:
      source: git::git@github.com:terraform-aws-modules/terraform-aws-dynamodb-table.git///
      version: v3.3.0
aws:
  region: us-east-1
tags:
  Managed-By: Terragrunt
  TF-Distribution: OpenTofu

Ensure that you have TERRAGRUNT_STATE_ENCRYPTION_PASSPHRASE environment variable setup. We will then run the apply command directly under the single-account/environments working directory:

terragrunt run-all apply -auto-approve

At this point the states are encrypted. We can now use the Terragrunt encryption block and remove the temporary generate block that we added to single-account/environments/terragrunt.hcl file:

locals {
  common_vars       = yamldecode(file("common.yaml"))
  state_bucket_name = yamldecode(file("global/environment.yaml")).module_config.state_bucket.bucket
  dynamodb_table    = yamldecode(file("global/environment.yaml")).module_config.state_locking_table.name
  state_encryption_passphrase = get_env("TERRAGRUNT_STATE_ENCRYPTION_PASSPHRASE")
}

# Configure Terragrunt to store state files in an S3 bucket
remote_state {
  backend = "s3"
  config = {
    bucket         = local.state_bucket_name
    key            = "${path_relative_to_include()}/state.json"
    region         = local.common_vars.aws.region
    dynamodb_table = local.dynamodb_table
  }

  encryption = {
    key_provider = "pbkdf2"
    passphrase   = local.state_encryption_passphrase
  }

  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
}

# Setup provider
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite"
  contents  = <<EOF
variable "aws_provider_default_tags" {
  type = map
}

provider "aws" {
  region = "${local.common_vars.aws.region}"
   default_tags {
    tags = var.aws_provider_default_tags
  }
}
EOF
}

inputs = {
  aws_provider_default_tags = local.common_vars.tags
}

And run a plan to see if everything works:

terragrunt run-all plan

So our old encryption configuration is still in there and OpenTofu refuses to add another one. Let's clean Terragrunt cache and run the plan again. Hopefully there will be no problems after that:

clean-terragrunt
terragrunt run-all plan

This asciinema cast is missing because I accidentally deleted it.

This concludes our migration from Terraform to OpenTofu. All state files are now encrypted with a passphrase, without which no one can decrypt our state files.


Removing the Warning

We have been encountering a Terragrunt warning in all of our Terragrunt runs during this blog post. As of version v0.70.0-beta2024120501 Terragrunt no longer considers root‑level file named terragrunt.hcl a best practice. This is because Terragrunt has also introduced Terragrunt Stacks which introduced the concept of shared and unit-specific Terragrunt config. I have yet to explore that. Maybe I’ll write a blog post for that too. For now we can safely move our root-level terragrunt.hcl file to root.hcl and see if Terragrunt complains about anything:

git mv terragrunt.hcl root.hcl
terragrunt run-all plan

I just wanted to see if Terragrunt would recognize root.hcl automatically. Since it doesn't do that automatically, we will update our downstream terragrunt.hcl files to explicitly mention the root-level Terragrunt config.

single-account/environments/global/state_bucket/terragrunt.hcl

include "root" {
  path = find_in_parent_folders("root.hcl")
}

include "s3_bucket" {
  path   = "../../_common/s3_bucket.hcl"
  expose = true
}

locals {
  module_vars = include.s3_bucket.locals.env_vars.module_config.state_bucket
}

inputs = merge(
  local.module_vars,
  {
    attach_policy = true
    policy = jsonencode(
      yamldecode(
        templatefile(
          "${get_repo_root()}/templates/policies/state_bucket_policy.yaml.tftpl",
          {
            bucket_name = local.module_vars.bucket
            account_id  = "${get_aws_account_id()}"
          }
        )
      )
    )
  }
)

single-account/environments/global/state_locking_table/terragrunt.hcl

include "root" {
  path = find_in_parent_folders("root.hcl")
}

include "dynamodb" {
  path   = "../../_common/dynamodb.hcl"
  expose = true
}

locals {
  module_vars = include.dynamodb.locals.env_vars.module_config.state_locking_table
}

inputs = merge(local.module_vars)

Let's try running the plan again:

terragrunt run-all plan

Wow! Now we have a whole new set of errors. After we made that small change in our downstream config files, Terragrunt can’t get the common variables because it cannot find the common.yaml file. I could try and spend more time diagnosing this to try and find the right path after multiple iterations of the plan command, but I am simply going to stop wasting my time on this with Terragrunt's get_repo_root() function in config files under the single-account/environments/_common directory.

single-account/environments/_common/dynamodb.hcl

locals {
  common_vars = yamldecode(file("${get_repo_root()}/single-account/environments/common.yaml")).terraform.remote_modules.dynamodb
  env_vars    = yamldecode(file("${path_relative_to_include()}/../environment.yaml"))

  module_source_url = local.common_vars.source
  module_version    = local.common_vars.version
}

terraform {
  source = "${local.module_source_url}?ref=${local.module_version}"
}

single-account/environments/_common/kms.hcl

locals {
  common_vars = yamldecode(file("${get_repo_root()}/single-account/environments/common.yaml")).terraform.remote_modules.kms
  env_vars    = yamldecode(file("${path_relative_to_include()}/../environment.yaml"))

  module_source_url = local.common_vars.source
  module_version    = local.common_vars.version
}

terraform {
  source = "${local.module_source_url}?ref=${local.module_version}"
}

single-account/environments/_common/s3_bucket.hcl

locals {
  common_vars = yamldecode(file("${get_repo_root()}/single-account/environments/common.yaml")).terraform.remote_modules.s3
  env_vars    = yamldecode(file("${path_relative_to_include()}/../environment.yaml"))

  module_source_url = local.common_vars.source
  module_version    = local.common_vars.version
}

terraform {
  source = "${local.module_source_url}?ref=${local.module_version}"
}

Let's try running the plan one more time:

terragrunt run-all plan

There we go. The warning is now removed.

The final PR for my changes is here.

This marks the end of this blog post. Your feedback is invaluable to me. Please share your thoughts and insights in the comments section below. I look forward to reading them!

Share

Tagged as: