Skip to content

Add an HCL store for .hcl and .tfvars files - #2323

Open
0x000NULL wants to merge 1 commit into
getsops:mainfrom
0x000NULL:feat/292
Open

0x000NULL wants to merge 1 commit into
getsops:mainfrom
0x000NULL:feat/292

Conversation

@0x000NULL

Copy link
Copy Markdown

Fixes #292

Adds an hcl store so Terraform .tfvars and Terragrunt-style .hcl files can be encrypted directly. It is picked by the .hcl/.tfvars extensions or --input-type/--output-type hcl; .tfvars.json stays JSON. It is built on hashicorp/hcl/v2 and wired in the same places as ini.

Design

  • Attributes are tree items. Strings, numbers and bools are encrypted with their type kept; lists and objects of literals become lists and mappings.
  • A block is keyed by its header including the brace, e.g. resource "aws_instance" "web" {. The { keeps tags { } distinct from tags = { }.
  • Any other expression (reference, function call, interpolated template, ...) is encrypted as its source text, wrapped as ${...} (HCL's own JSON convention), and written back verbatim on decrypt. I encrypt these rather than leave them readable because an expression can hold a secret, and a plaintext leaf would need core changes. --unencrypted-regex still works as usual.
  • Comments are kept as sops comments. Metadata is a top-level sops = { ... } attribute.

Limitations

  • Output is re-emitted in terraform fmt style: blank lines are dropped, heredocs become quoted strings, 1.0 becomes 1.
  • Inline comments come back on the next line after decrypt. This is an existing core issue that also affects yaml.
  • Comments inside an attribute or block header (k /* c */ = "v") are dropped.
  • Strings that HCL would normalize (non-NFC text or \u escapes) are kept as source text so they round-trip byte-exact; such block labels are rejected.
  • A literal string shaped like ${...} is rejected, since it can't be told apart from an expression.
  • Paths for set/unset/--extract use the full block header in single quotes: ['resource "aws_instance" "web" {']["ami"].

Dependency

hcl/v2 and go-cty direct, plus 5 indirect modules. The linux/amd64 binary grows by 2.4 MB (3.2%).

Testing

  • Unit tests for stores/hcl cover round trips, blocks, expressions, comments, Unicode, error cases, and a full AES encrypt/decrypt that checks no secret or expression text is visible. The formats tests and the decrypt_format functional test now cover HCL too.
  • go test ./...: only hcvault and kms fail (no Docker socket), the same as on main. go vet and make vendor && git diff --exit-code are clean.
  • Manually with age: a .hcl file with nested and repeated labeled blocks decrypts byte-identical. A tampered ENC[...] value fails the MAC check. terraform plan -var-file on an encrypted .tfvars only warns about the undeclared sops variable.

Add a store that reads and writes HCL, so Terraform .tfvars and
Terragrunt .hcl files can be encrypted directly. The format is selected
by the .hcl and .tfvars extensions or with --input-type/--output-type hcl.

Attributes and (labeled) blocks are mapped to the sops tree. A block is
stored under its header, for example `resource "aws_instance" "web" {`,
which keeps it distinct from an attribute holding an object. Literal
strings, numbers and bools are encrypted, as are lists and objects made
of literals. Any other expression (references, function calls,
interpolated templates, ...) is kept as its source text, wrapped as
`${...}`, and encrypted like a string, so it is written back verbatim
on decryption. Comments are kept; the sops metadata is stored as a
top-level `sops` attribute.

Fixes getsops#292

Signed-off-by: Ethan Aldrich <ethan@ethanaldrich.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support HCL files

1 participant