Skip to content

Explanation

Scope

How Terraform works and why it is built this way: core engine internals, the provider plugin protocol, plan/apply and state, the newer language features (ephemeral values, tests, Stacks, query, actions), the HCP Terraform / Terraform Enterprise platform, licensing and ownership history, and the security model. Look-up tables are in Reference; tasks are in How-to Guides; the hub is Terraform.

Terraform is HashiCorp's (IBM's since 2025-02-27) infrastructure-as-code tool. Users describe resources in HCL; the core engine parses the configuration, builds a dependency graph, and drives provider plugins over gRPC to reconcile desired state with real infrastructure, recording the result in a state file.

Component Overview

The diagram shows the CLI's internal layers, the plugin boundary, state backends, registries, and the optional HCP Terraform / Terraform Enterprise platform.

graph TB
    subgraph CLI["terraform CLI (Go, BSL 1.1)"]
        TF["Command layer<br/>init, plan, apply, test, query, stacks"]
    end

    subgraph CORE["Terraform Core"]
        HCL["HCL parser<br/>(hashicorp/hcl/v2)"]
        CFG["Config loader<br/>(module installer)"]
        GRAPH["Graph builder<br/>(DAG + transformers)"]
        EVAL["Evaluator<br/>(expressions, count, for_each)"]
        PLAN["Plan engine"]
        APPLY["Apply engine"]
    end

    subgraph PLUGINS["Plugin processes"]
        PV["Provider plugins<br/>(gRPC tfplugin5 / tfplugin6)"]
        PROV["Provisioners<br/>(file, local-exec, remote-exec)"]
    end

    subgraph STATE["State layer"]
        SF["State snapshot<br/>(JSON, format v4)"]
        BE["Backend<br/>(s3, gcs, azurerm, oci, cloud)"]
        WS["Workspaces"]
    end

    subgraph REG["Registries"]
        TREG["registry.terraform.io"]
        PRIV["Private registry<br/>(HCP Terraform / TFE)"]
    end

    subgraph HCPTF["HCP Terraform / Terraform Enterprise (optional)"]
        RUNS["Remote runs + agents"]
        POL["Sentinel / OPA policies"]
        STK["Stacks orchestration"]
    end

    TF --> HCL --> CFG --> GRAPH --> EVAL --> PLAN --> APPLY
    APPLY --> PV
    APPLY --> PROV
    PLAN --> SF
    APPLY --> SF
    SF --> BE
    WS --> SF
    CFG -->|"modules + providers"| TREG
    CFG -->|"modules + providers"| PRIV
    TF -->|"cloud block / remote ops"| HCPTF

Core Components

HCL Parser

The HCL parser (Go library github.com/hashicorp/hcl/v2) handles:

  • Parsing .tf files (and .tf.json) into an abstract syntax tree.
  • Evaluating HCL expressions: variables, locals, conditionals, for expressions, and template strings.
  • Validating block types and attributes against provider-supplied schemas.
  • Producing diagnostics with source locations.

The same library is used by other HashiCorp tools (Vault, Consul, Nomad, Packer), which is why their configuration looks alike.

Config Loader and Module System

The config loader assembles the full configuration tree:

  • Root module: the .tf files in the working directory.
  • Child modules: referenced via module blocks, sourced from local paths, the Terraform Registry, Git repositories, S3/GCS buckets, HTTP archives, or an HCP Terraform / TFE private registry.

Since 1.15, source and version in module blocks may use variables and locals, so module sources can be chosen per environment instead of being hard-coded; most commands therefore now accept variable values. Provider versions and checksums are pinned in .terraform.lock.hcl; module versions are not locked there and must be pinned in the module block.

Graph Builder

The graph builder constructs a directed acyclic graph (DAG) that determines operation order:

  • Nodes: root and module variables, providers, resources, data sources, ephemeral resources, outputs, locals, actions.
  • Edges: implicit dependencies discovered from expression references, plus explicit depends_on.
  • Transformers: a pipeline of graph transformers shapes the DAG, for example ReferenceTransformer (connects references to targets), ProviderTransformer (associates resources with provider instances), OrphanResourceTransformer (finds resources in state but not in config), and TransitiveReductionTransformer (removes redundant edges).

Each operation has its own builder (PlanGraphBuilder, ApplyGraphBuilder, and graphs for validate and evaluation walks); destroy and refresh-only runs are plan modes on the plan graph. Resources without a path between them are walked in parallel, bounded by -parallelism (default 10).

Evaluator

The evaluator walks the graph and resolves all expressions:

  • Evaluates variable values from .tfvars files, TF_VAR_* environment variables, and CLI flags.
  • Resolves locals in dependency order.
  • Expands count and for_each into resource instances (unknown values here are an error unless the experimental deferred-actions mode is used).
  • Applies lifecycle meta-arguments (create_before_destroy, prevent_destroy, ignore_changes, replace_triggered_by, and destroy = false since 1.16).
  • Tracks value "marks" such as sensitive and ephemeral, which decide where a value may flow.

Plugin Protocol

Terraform talks to providers over a gRPC plugin protocol. Protocol 5 (tfplugin5) arrived with 0.12; protocol 6 (tfplugin6) added nested attribute types and is what terraform-plugin-framework speaks. Minor protocol versions add optional capabilities (functions, ephemeral resources, write-only attributes, list resources, actions) that core discovers through feature detection. Version and SDK tables are in Reference.

Plugin Handshake

The sequence shows how core launches a provider and drives it through a plan and apply.

sequenceDiagram
    participant Core as Terraform Core
    participant Plugin as Provider plugin process

    Core->>Plugin: Launch child process (go-plugin)
    Plugin-->>Core: Handshake line on stdout<br/>protocol version + address + TLS cert
    Core->>Plugin: Open gRPC connection (mTLS)
    Core->>Plugin: GetProviderSchema
    Plugin-->>Core: Resource, data source, function schemas
    Core->>Plugin: ConfigureProvider
    Plugin-->>Core: Configured
    Note over Core,Plugin: Plan phase
    Core->>Plugin: ReadResource (refresh)
    Core->>Plugin: PlanResourceChange (per instance)
    Plugin-->>Core: Planned state + requires_replace
    Note over Core,Plugin: Apply phase
    Core->>Plugin: ApplyResourceChange (per instance)
    Plugin-->>Core: New state
    Core->>Plugin: StopProvider / kill process

Plan / Apply Flow

The flowchart follows one init / plan / apply cycle, including where state is read and written.

flowchart TD
    A["terraform init"] --> B["Install providers<br/>Download modules<br/>Initialize backend"]
    B --> C["terraform plan"]
    C --> D["Parse HCL config"]
    D --> E["Load prior state from backend"]
    E --> F["Refresh: read real-world objects via providers"]
    F --> G["Evaluate config against refreshed state"]
    G --> H["Build plan graph"]
    H --> I["Compute diff per resource instance"]
    I --> J["Save plan file (optional)"]
    J --> K["terraform apply"]
    K --> L["Load saved plan OR re-plan"]
    L --> M["Build apply graph"]
    M --> N["Walk graph: create / update / delete"]
    N --> O["Update state incrementally"]
    O --> P["Persist final state to backend"]
    P --> Q["terraform output"]

Saved plans are applied exactly

A saved plan file is a zip archive holding the planned changes plus a snapshot of the configuration and prior state. terraform apply plan.tfplan executes only those changes and refuses a stale plan if the state changed since planning (serial/lineage check). Since 1.15 it also refuses a plan made for a different workspace. Plan files can contain sensitive values, so treat them like state.

How It Works

Core Lifecycle

The sequence shows what an operator's plan and apply do between the CLI, state, a provider, and the cloud API.

sequenceDiagram
    participant Op as Operator
    participant CLI as Terraform CLI
    participant State as State backend
    participant Provider as Provider plugin (gRPC)
    participant Cloud as Cloud API

    Op->>CLI: terraform plan
    CLI->>State: Read prior state (and lock)
    CLI->>Provider: ReadResource for each tracked object
    Provider->>Cloud: Describe / Get calls
    Provider-->>CLI: Current attributes
    CLI->>CLI: Diff desired (HCL) vs current
    CLI-->>Op: Execution plan (create, update, replace, destroy)

    Op->>CLI: terraform apply
    CLI->>CLI: Build apply graph (DAG)
    CLI->>Provider: ApplyResourceChange
    Provider->>Cloud: Create / Update / Delete calls
    Provider-->>CLI: New attributes
    CLI->>State: Write updated state (and unlock)

Dependency Graph (DAG)

Terraform builds a directed acyclic graph of all objects; this example shows how references on AWS resources become ordering edges.

flowchart TB
    VPC["aws_vpc.main"] --> Subnet["aws_subnet.app"]
    VPC --> SG["aws_security_group.web"]
    Subnet --> Instance["aws_instance.web"]
    SG --> Instance
    Instance --> EIP["aws_eip.web"]

aws_subnet.app and aws_security_group.web have no edge between them, so they are created in parallel. Destroys walk the same graph in reverse.

Provider Plugin Architecture

Each provider is a separate binary downloaded by terraform init from a registry (or a filesystem/network mirror) and run as a child process.

flowchart LR
    CORE["terraform core"] <-->|"gRPC (muxed SDKv2 + framework)"| AWS["hashicorp/aws"]
    CORE <-->|"gRPC"| GOOG["hashicorp/google"]
    CORE <-->|"gRPC"| K8S["hashicorp/kubernetes"]
    CORE <-->|"gRPC tfplugin6"| CUSTOM["Custom provider<br/>(plugin-framework)"]
    LOCK[".terraform.lock.hcl<br/>(versions + checksums)"] --> CORE

Because providers are separate processes with a versioned wire protocol, they release independently of core, and OpenTofu can run the same provider binaries.

State Management

State File Structure

The state (terraform.tfstate) is a JSON document with:

  • version: state format version (currently 4).
  • terraform_version: the version that last wrote it.
  • serial: counter incremented on every write; used to detect stale plans and conflicting writes.
  • lineage: UUID fixed at state creation; prevents applying one state's plan against a different state.
  • outputs: root module outputs.
  • resources: resource instances with type, name, provider, and per-instance attributes, dependencies, schema_version, and (since 1.12) resource identity.

State exists because many cloud APIs cannot answer "which objects did this configuration create?". It maps configuration addresses to remote object IDs and caches attributes, which is also why it can contain secrets. Backend options are compared in Reference.

Workspaces

CLI workspaces are multiple named state snapshots for one configuration and one backend:

  • Each workspace has its own state; default always exists.
  • terraform.workspace exposes the name for conditional logic.
  • They suit near-identical copies (for example, ephemeral feature environments); environments that differ in structure or credentials are better as separate root modules or Stacks deployments.
  • HCP Terraform "workspaces" are a different, heavier concept: each has its own state, variables, VCS trigger, run queue, and permissions.

Ephemeral Values and Write-only Attributes

Secrets in state were Terraform's longest-standing security complaint. Since 1.10, values can be marked ephemeral: they exist only during a single plan or apply and are never written to plan files or state.

Feature Since What it is
ephemeral resources 1.10 Read fresh in each phase (for example a Vault secret or a generated password); never persisted
ephemeral = true on variables/outputs 1.10 Values that may only flow into ephemeral contexts
ephemeralasnull() 1.10 Strips ephemeral values to null where a persisted value is required
Write-only attributes (*_wo) 1.11 Provider arguments that accept ephemeral values and are not stored in state; a companion *_wo_version triggers updates
terraform_data store block 1.16 Carries ephemeral/sensitive values between plan and apply
mock_provider for ephemeral resources 1.17 (beta) Lets tests mock ephemeral resources

The flow below shows why nothing lands in state: the secret is opened, passed to a write-only argument, and closed within the same run.

sequenceDiagram
    participant Core as Terraform Core
    participant Eph as Ephemeral resource<br/>(e.g. vault secret)
    participant Res as Managed resource<br/>(write-only argument)
    participant St as State backend

    Core->>Eph: OpenEphemeralResource
    Eph-->>Core: Secret value (marked ephemeral)
    Core->>Res: ApplyResourceChange with password_wo
    Res-->>Core: New state (write-only value set to null)
    Core->>Eph: CloseEphemeralResource
    Core->>St: Persist state without the secret

Provider support is required: the provider must implement the ephemeral resource or the write-only argument (framework-based providers only).

Testing Model

terraform test (GA in 1.6) runs *.tftest.hcl files. Each file contains run blocks that execute plan or apply against the module under test and check assert conditions; real infrastructure created by apply runs is destroyed at the end in reverse run order.

  • Mocks (1.7): mock_provider, override_resource, override_data, override_module let tests run without cloud credentials; 1.15 allows functions inside mock blocks.
  • Scale (1.12-1.13): -parallelism, parallel-eligible runs, parallel teardown, and variable definitions inside test files.
  • CI integration (1.11): -junit-xml output is GA.
  • Experimental: test backend blocks, skip_cleanup, and terraform test cleanup keep long-lived fixtures (alpha builds only).

Unit-style tests (plan + mocks) and integration tests (apply against a sandbox account) share the same syntax; the choice is per run block via command.

Terraform Stacks

Stacks (GA in HCP Terraform at HashiConf, 2025-09-25) add a layer above root modules. A Stack is a set of components (module calls, in *.tfcomponent.hcl) deployed together into several deployments (for example per region or environment, in *.tfdeploy.hcl). HCP Terraform plans all components of a deployment as one unit, orders them by their dependencies, and can defer components whose inputs are unknown until upstream components apply.

The diagram shows one Stack configuration fanned out to several deployments, with a linked Stack consuming published outputs.

flowchart LR
    subgraph CFG["Stack configuration (VCS repo)"]
        C1["component vpc<br/>(*.tfcomponent.hcl)"]
        C2["component eks"]
        C3["component apps"]
        D["deployments<br/>(*.tfdeploy.hcl)"]
    end
    subgraph HCP["HCP Terraform Stacks"]
        DEV["deployment dev"]
        PRODUS["deployment prod-us"]
        PRODEU["deployment prod-eu"]
    end
    LINK["Linked Stack<br/>(upstream_input)"]
    C1 --> C2 --> C3
    D --> DEV
    D --> PRODUS
    D --> PRODEU
    PRODUS -->|"publish_output"| LINK

What changed at GA: the separate terraform-stacks-cli was deprecated in favor of terraform stacks (in core since 1.13), Stack APIs gained backward-compatibility guarantees, configuration files were renamed (.tfstack.hcl to .tfcomponent.hcl), and Stacks resources started counting toward RUM billing. Stacks require an RUM-based HCP Terraform plan or Terraform Enterprise 2.0+ (see update to GA).

Search, Actions, and Policy

Three features extend Terraform beyond the create/read/update/delete model:

Feature CLI HCP Terraform Idea
List resources + terraform query GA in 1.14 Terraform Search public beta (2025-09) Providers enumerate existing objects via list blocks in *.tfquery.hcl; results can be turned into import blocks and config for bulk import
Actions (action block, -invoke) GA in 1.14; on_failure, destroy events in 1.16 Public beta (2025-09) Provider-defined day-2 operations (for example Lambda invoke, CloudFront invalidation, Ansible playbook) triggered from resource lifecycle events or on demand
Terraform Policy (-policies) Experimental 1.16; GA in 1.17 (beta) Policy results shown on HCP runs since 1.16 Evaluate policy sets locally against plans, applies, and query results

HCP Terraform and Terraform Enterprise

HCP Terraform (named Terraform Cloud until 2024) is the SaaS platform; Terraform Enterprise (TFE) is its self-managed distribution. Both run the same CLI remotely and add:

Feature Description
Remote operations Plans and applies run on HashiCorp-managed workers or self-hosted HCP Terraform agents
VCS integration Runs triggered by pushes and pull requests (GitHub, GitLab, Bitbucket, Azure DevOps)
Policy as code Sentinel and OPA policy sets with advisory / soft- / hard-mandatory enforcement
Private registry Organization modules and providers; no-code modules
Stacks Multi-component, multi-deployment orchestration
Dynamic provider credentials OIDC workload identity tokens for AWS, Azure, GCP, Vault, Kubernetes
HYOK Hold Your Own Key encryption of state and plan artifacts (GA 2025-09)
Run tasks Third-party checks (cost, security scanning) inserted into runs
RBAC and SSO Teams, projects, workspace permission sets, SAML/OIDC SSO
API Full REST API; hashicorp/tfe provider manages HCP Terraform with Terraform

Pricing moved fully to resources under management (RUM) after the legacy Free plan ended on 2026-03-31; plan prices and TFE versioning are in Reference.

Licensing and Governance

The timeline shows the license and ownership events that shape how Terraform is adopted today.

timeline
    title Terraform license and ownership
    2014 : Terraform 0.1 released under MPL 2.0
    2023-08 : HashiCorp announces move to BSL 1.1
    2023-09 : 1.5.7 is the last MPL release, OpenTofu fork follows
    2023-10 : 1.6.0 is the first BSL release
    2024-04 : Terraform Cloud renamed HCP Terraform
    2025-02 : IBM closes HashiCorp acquisition
    2025-12 : CDKTF archived
    2026-03 : Legacy HCP Terraform Free plan retired
  • License: Terraform 1.6.0 and later are under the Business Source License 1.1. The licensor is now IBM Corp. Production use is allowed unless you offer Terraform to third parties, on a hosted or embedded basis, in a paid product that significantly overlaps with IBM's paid Terraform versions. Each version converts to MPL 2.0 four years after publication (so 1.6.0, published 2023-10-04, converts in 2027-10). Providers such as hashicorp/aws and libraries such as HCL remain MPL 2.0. Source: LICENSE, license FAQ.
  • Ownership: IBM completed its acquisition of HashiCorp on 2025-02-27 (about $6.4B enterprise value, $35 per share). Visible product changes since then: CDKTF sunset, the legacy Free plan retired, TFE moved to quarterly semantic-versioned releases, and license and copyright text now name IBM.
  • OpenTofu: The Linux Foundation fork of 1.5.x (now CNCF) stays MPL 2.0 and has diverged with features such as client-side state encryption; see OpenTofu.
  • CDKTF: Terraform CDK was archived on 2025-12-10 (last release 0.21.0 on 2025-06-04). HashiCorp recommends cdktf synth --hcl to migrate to plain HCL, or AWS CDK for AWS-centric users. See hashicorp/terraform-cdk.

Security Model

Authentication Model

Terraform itself does not authenticate users; the platform, the state backend, and each provider do:

Component Authentication method
HCP Terraform / TFE User accounts, SSO (SAML/OIDC), user/team/organization API tokens
AWS provider Static keys, IAM roles, AssumeRoleWithWebIdentity (OIDC)
Azure provider Service principals, managed identity, OIDC federated credentials
Google provider Service accounts, Workload Identity Federation
State backends (S3, GCS, Azure) Backend-specific credentials (S3 backend also supports aws login since 1.15)

The diagram shows the identity path from operators through HCP Terraform to cloud providers with dynamic credentials.

graph TB
    subgraph Operators
        Dev["Developer<br/>(terraform CLI)"]
        VCS["VCS webhook<br/>(GitHub / GitLab)"]
        CI["CI pipeline<br/>(team API token)"]
    end
    subgraph TFC["HCP Terraform / TFE"]
        AuthN["Identity<br/>(SSO, API tokens)"]
        Run["Run worker or agent"]
        Pol["Sentinel / OPA<br/>policy checks"]
        StateStore["State storage<br/>(encrypted, optional HYOK)"]
        OIDC["Workload identity token<br/>(per run, signed JWT)"]
    end
    subgraph Clouds
        AWS["AWS STS<br/>AssumeRoleWithWebIdentity"]
        Azure["Azure AD<br/>federated credential"]
        GCP["GCP STS<br/>Workload Identity Federation"]
    end
    Dev --> AuthN
    VCS --> AuthN
    CI --> AuthN
    AuthN --> Run
    Run --> Pol
    Run --> StateStore
    Run --> OIDC
    OIDC --> AWS
    OIDC --> Azure
    OIDC --> GCP

State File Risks

The state file is the most sensitive Terraform artifact: it holds every managed object ID, computed attributes, and any secret that passed through a non-ephemeral attribute.

Risk Impact
Plaintext secrets Database passwords, keys, and tokens readable by anyone who can read state
Resource manipulation Edited state can cause unintended destroys or hide drift
Sensitive outputs Internal IPs, DNS names, instance IDs
Drift concealment Altered state masks unauthorized changes

sensitive = true only redacts CLI output; it does not keep values out of state. Ephemeral values and write-only attributes are the mechanisms that do.

HCP Terraform State Security

HashiCorp's data security page (last modified 2026-07-31, checked 2026-09-27) says state files and plan results live in blob storage and are encrypted with Vault Transit. Each object gets its own 128-bit AES-GCM key. That key is wrapped by the Vault transit engine (AES-GCM, 256-bit key) and stored next to the object (envelope encryption). HCP Terraform rotates the root keys every 365 days; Terraform Enterprise does not rotate them automatically. Audit trails and organization, workspace and team settings are stored unencrypted in PostgreSQL. Every state write is kept as a state version, so you can roll back. Hold your own key (HYOK) makes the HCP Terraform agent encrypt state and plan data with a key from your own KMS before it uploads them.

Policy as Code

Sentinel and OPA policies run between plan and apply on HCP Terraform / TFE, reading the plan, configuration, and state as structured data. Enforcement level decides whether a failing policy warns, blocks until overridden, or blocks outright (see Reference). Terraform Policy (GA in 1.17) brings policy evaluation to the CLI via -policies.

Dynamic Credentials

Static cloud keys in workspace variables are long-lived and hard to rotate. With dynamic provider credentials, HCP Terraform mints a signed workload identity token per run; the cloud's STS exchanges it for short-lived credentials scoped by a trust policy that can match organization, project, workspace, and run phase (plan vs apply). A leaked token expires within the run's lifetime. Setup steps are in How-to Guides.

Comparison with Alternatives

Aspect Terraform OpenTofu Pulumi
Language HCL HCL TypeScript, Python, Go, .NET, Java, YAML, HCL (Pulumi HCL)
License BSL 1.1 (IBM) MPL 2.0 Apache 2.0 (engine)
Provider protocol gRPC tfplugin5/6 Same (compatible) Own gRPC; bridges Terraform providers
State encryption Backend at rest; HYOK on HCP Built-in client-side (AES-GCM + KMS) Per-secret encryption
Policy as code Sentinel / OPA (HCP, TFE); Terraform Policy CLI (1.17) OPA via external tools CrossGuard
Managed platform HCP Terraform / TFE Third-party (Spacelift, env0, Scalr, and others) Pulumi Cloud

See IaC Comparison for the full matrix and decision flowchart.

Benchmarks

Terraform publishes no official performance benchmarks. Rough, unsourced estimates for plan/apply time by state size, module count, and provider rate limits are kept in Reference, clearly flagged as estimates. Practical levers (splitting state, -parallelism, provider plugin caching, -minimal-refresh in 1.17) are in How-to Guides.

Sources