Can't sign GHA cache blobs on pull request from a fork

[crazy-max]

ACTIONS_ID_TOKEN_REQUEST_TOKEN and ACTIONS_ID_TOKEN_REQUEST_URL vars are not set when a pull request is made from a fork: https://github.com/docker/github-builder-experimental/actions/runs/20853632295/job/59914492700#step:6:215

Driver Options:        env.ACTIONS_ID_TOKEN_REQUEST_TOKEN="" env.ACTIONS_ID_TOKEN_REQUEST_URL="" image="moby/buildkit:master@sha256:bdefeba47634c596286beabe68219708ed364c4f1a5e4e9a2e160274712a0e89"

https://github.com/docker/github-builder-experimental/actions/runs/20853632295/job/59914492700#step:5:1

ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE=/opt/actionarchivecache
ACTIONS_RUNTIME_URL=https://pipelinesghubeus5.actions.githubusercontent.com/Gn0SFRdzfC2QQgKXjTaZ1iG2iqr70vWtXE5esVcflAtH7dtmQI/
ACTIONS_RUNTIME_TOKEN=***
ACTIONS_CACHE_URL=https://artifactcache.actions.githubusercontent.com/Gn0SFRdzfC2QQgKXjTaZ1iG2iqr70vWtXE5esVcflAtH7dtmQI/
ACTIONS_RESULTS_URL=https://results-receiver.actions.githubusercontent.com/
ACTIONS_CACHE_SERVICE_V2=True

I can't find any documentation about this:

• https://docs.github.com/en/actions/how-tos/secure-your-work/security-harden-deployments/oidc-with-reusable-workflows
• https://docs.github.com/en/actions/concepts/security/openid-connect
• https://docs.github.com/en/actions/reference/security/oidc

But it seems to be the same strategy that is applied to the GITHUB_TOKEN.

That breaks GHA cache with our reusable workflow unfortunately: https://github.com/docker/github-builder-experimental/actions/runs/20853632295/job/59914492700#step:10:343

#16 exporting to GitHub Actions Cache
#16 preparing build cache for export
#16 writing layer sha256:12235759f37b93f211a3e31e2293a471d186598e42a53df9cc3532d232c61876
#16 writing layer sha256:12235759f37b93f211a3e31e2293a471d186598e42a53df9cc3532d232c61876 0.2s done
#16 writing layer sha256:8e62030b6c098dc055738c3b4a87c658c881a152b8b8d6c391db551deeda8bcd
#16 writing layer sha256:8e62030b6c098dc055738c3b4a87c658c881a152b8b8d6c391db551deeda8bcd 0.2s done
#16 signing cache index sha256:7ce6816147a0f2ea5a99992d35dbe90b18b2131fbeb9038df40a08eb7a5041c6
#16 preparing build cache for export 305.1s done
#16 signing cache index sha256:7ce6816147a0f2ea5a99992d35dbe90b18b2131fbeb9038df40a08eb7a5041c6 304.2s done
#16 ERROR: signing command failed: + cosign sign-blob --yes --oidc-provider github-actions --new-bundle-format --use-signing-config --bundle /tmp/tmp.KICOOH '--tlog-upload=false' /tmp/tmp.DHADPH
Non-interactive mode detected, using device flow.
Error: signing /tmp/tmp.DHADPH: retrieving ID token: authenticating caller: error obtaining token: expired_token
error during command execution: signing /tmp/tmp.DHADPH: retrieving ID token: authenticating caller: error obtaining token: expired_token
+ rm -f /tmp/tmp.DHADPH /tmp/tmp.KICOOH
: exit status 1

[crazy-max]

Also it seems to hang for 5m when cosign tries to sign without an OIDC token available:

305.1s

#16 exporting to GitHub Actions Cache
#16 preparing build cache for export
#16 writing layer sha256:12235759f37b93f211a3e31e2293a471d186598e42a53df9cc3532d232c61876
#16 writing layer sha256:12235759f37b93f211a3e31e2293a471d186598e42a53df9cc3532d232c61876 0.2s done
#16 writing layer sha256:8e62030b6c098dc055738c3b4a87c658c881a152b8b8d6c391db551deeda8bcd
#16 writing layer sha256:8e62030b6c098dc055738c3b4a87c658c881a152b8b8d6c391db551deeda8bcd 0.2s done
#16 signing cache index sha256:7ce6816147a0f2ea5a99992d35dbe90b18b2131fbeb9038df40a08eb7a5041c6
#16 preparing build cache for export 305.1s done
#16 signing cache index sha256:7ce6816147a0f2ea5a99992d35dbe90b18b2131fbeb9038df40a08eb7a5041c6 304.2s done
#16 ERROR: signing command failed: + cosign sign-blob --yes --oidc-provider github-actions --new-bundle-format --use-signing-config --bundle /tmp/tmp.KICOOH '--tlog-upload=false' /tmp/tmp.DHADPH
Non-interactive mode detected, using device flow.
Error: signing /tmp/tmp.DHADPH: retrieving ID token: authenticating caller: error obtaining token: expired_token
error during command execution: signing /tmp/tmp.DHADPH: retrieving ID token: authenticating caller: error obtaining token: expired_token
+ rm -f /tmp/tmp.DHADPH /tmp/tmp.KICOOH
: exit status 1

Was thinking it could be related to default cosign timeout:

  -t, --timeout duration     timeout for commands (default 3m0s)

But still hangs for 5m https://github.com/docker/github-builder-experimental/actions/runs/20855388353/job/59920552605#step:10:465 after setting timeout to 10s: 4164eae

#17 exporting to GitHub Actions Cache
#17 preparing build cache for export
#17 writing layer sha256:4e13f9c206f9fccf33d9ac498408b1516509db8e529277ea1072802d247071b2
#17 writing layer sha256:4e13f9c206f9fccf33d9ac498408b1516509db8e529277ea1072802d247071b2 0.5s done
#17 writing layer sha256:be95b65492a2a1ce7ec9f227f54fb8b0cf8ac7027733d067ac8530bb55426794
#17 writing layer sha256:be95b65492a2a1ce7ec9f227f54fb8b0cf8ac7027733d067ac8530bb55426794 0.5s done
#17 signing cache index sha256:c6da9f323f405df5eeee899a960a8324a7e98eae686a8c1ad3ecd991cad1c3ca
#17 preparing build cache for export 302.7s done
#17 signing cache index sha256:c6da9f323f405df5eeee899a960a8324a7e98eae686a8c1ad3ecd991cad1c3ca 300.8s done
#17 ERROR: signing command failed: + cosign sign-blob --yes --timeout 10s --oidc-provider github-actions --new-bundle-format --use-signing-config --bundle /tmp/tmp.HOFIpO '--tlog-upload=false' /tmp/tmp.ajOnpO
Non-interactive mode detected, using device flow.
Error: signing /tmp/tmp.ajOnpO: retrieving ID token: authenticating caller: error obtaining token: expired_token
error during command execution: signing /tmp/tmp.ajOnpO: retrieving ID token: authenticating caller: error obtaining token: expired_token
+ rm -f /tmp/tmp.ajOnpO /tmp/tmp.HOFIpO
: exit status 1

@haydentherapper Is it a known issue?

[Hayden-IO]

Hm, surprised to see it hanging. We do see GHA periodically not having an OIDC token available, but I'd expect it to timeout faster. Possibly we aren't passing the timeout through the contexts properly.

[crazy-max]

Hm, surprised to see it hanging. We do see GHA periodically not having an OIDC token available, but I'd expect it to timeout faster. Possibly we aren't passing the timeout through the contexts properly.

I will take a look if I repro locally and keep you posted

Can't sign GHA cache blobs on pull request from a fork

Fixed in #77 by checking if OIDC token available otherwise skip signing cache as blob storage is scoped to the pull request anyway.