Skip to content

Match .gitignore specs relative to walk root in walk_with_gitignore - #210

Merged
mgrohols merged 1 commit into
awsdocs:mainfrom
brmur:fix/walk-gitignore-relative-path
Oct 15, 2026
Merged

mgrohols merged 1 commit into
awsdocs:mainfrom
brmur:fix/walk-gitignore-relative-path

Conversation

@brmur

@brmur brmur commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem

walk_with_gitignore (in aws_doc_sdk_examples_tools/file_utils.py) matches .gitignore specs against each file's absolute path. Git evaluates .gitignore patterns relative to the repository root, so a root-anchored pattern (e.g. /local, /home, /tmp) matches the leading segment of the checkout's absolute path. On a checkout under /local/home/<user>/aws-doc-sdk-examples (the default on many internal Cloud Desktops), a /local entry in any .gitignore matches /local/... and silently excludes the entire subtree from the walk.

Real-world instance

In awsdocs/aws-doc-sdk-examples, kotlin/services/bedrock/.gitignore was copied verbatim from the upstream JetBrains Kotlin compiler repo and contained a root-anchored /local pattern. On a /local/... checkout this dropped all of kotlin/services/bedrock/src from the WRITEME file walk, so every snippet beneath src/ reported as missing on every run:

DocGen Error: In .doc_gen/metadata/bedrock_metadata.yaml at Kotlin:1,
missing snippet bedrock.kotlin.list_foundation_models.main

That was patched repo-side by replacing the stray .gitignore in awsdocs/aws-doc-sdk-examples#8213, but that only addresses one file. This PR fixes the underlying tool bug so the whole class of root-anchored-pattern collisions can't recur, for any .gitignore in any consuming repo.

(The symptom does not appear on GitHub-hosted CI, whose runner paths are /home/runner/work/... and never match /local; it only manifests on local /local/... checkouts.)

Fix

Thread the top-level walk root through the recursion as base and match each path relative to it, mirroring git's semantics. Root-anchored patterns still work correctly relative to the walk root (a /build pattern ignores top-level build/ but keeps a nested src/build/).

Tests

Added two regression tests to file_utils_test.py:

  • test_root_anchored_pattern_does_not_match_checkout_prefix — reproduces the /local collision; fails on the old code, passes with the fix.
  • test_root_anchored_pattern_still_ignores_matching_dir — confirms root-anchored patterns are still honored relative to the walk root (not merely disabled).

All 110 tests pass (python3 -m pytest).

Verified on a real /local checkout

Beyond the mocked unit tests, I exercised the fix against a real on-disk aws-doc-sdk-examples checkout under /local/... (the condition that triggers the bug), through the real snippet-scanning code path (collect_snippets -> get_files -> walk_with_gitignore):

  • Real walk_with_gitignore on an on-disk tree with a root-anchored /local pattern: the fixed code yields the Kotlin source file and still ignores build/; the pre-fix absolute-matching logic dropped the entire tree.
  • Real collect_snippets on the actual repo, with /local temporarily injected into kotlin/services/bedrock/.gitignore: fixed code finds bedrock.kotlin.list_foundation_models.main; pre-fix absolute-matching does not find it — exactly the [Kotlin] Replace JetBrains boilerplate in bedrock .gitignore aws-doc-sdk-examples#8213 symptom.
  • Clean-tree writeme.py --check for Kotlin bedrock (no /local): passes with no missing-snippet / DocGen errors.

The injected .gitignore edit was temporary and restored to its clean tracked state after each test.

Related

walk_with_gitignore matched .gitignore specs against each file's
absolute path. Git evaluates patterns relative to the repository
root, so a root-anchored pattern (e.g. `/local`) matched the leading
segment of an absolute checkout path such as
`/local/home/<user>/aws-doc-sdk-examples` and silently excluded every
file beneath the walk root.

For WRITEME this meant a service's entire `src/` tree was dropped from
the file walk on any `/local/...` checkout (the Cloud Desktop default),
so every snippet beneath it reported as missing on every run,
regardless of the author's change.

Thread the top-level walk root through the recursion as `base` and
match each path relative to it, mirroring git. Root-anchored patterns
continue to work relative to the walk root (verified: `/build` ignores
top-level build/ but keeps nested src/build/).

A repo-side workaround for the specific `/local` collision was shipped
in aws-doc-sdk-examples PR #8213; this fixes the underlying tool bug
and prevents the whole class of issue.
@mgrohols
mgrohols merged commit 044ca09 into awsdocs:main Oct 15, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Enhancement A general update to the code base.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants