Skip to content

test(e2e): self-contained Keycloak for e2e stack - #187

Open
gcgoncalves wants to merge 3 commits into
mainfrom
6910-sso-e2e-infra
Open

gcgoncalves wants to merge 3 commits into
mainfrom
6910-sso-e2e-infra

Conversation

@gcgoncalves

@gcgoncalves gcgoncalves commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Closes IBM/mcp-context-forge#6926

Add a keycloak service and e2e/keycloak/realm-export.json to docker-compose.e2e.yml so this repo's e2e suite never needs a sibling mcp-context-forge checkout to exercise SSO. The realm seeds two clients (mcp-gateway for the real gateway image, contextforge-web-ui for this repo's own BFF) and one test user, modeled on (not synced with) mcp-context-forge's own infra/keycloak/realm-export.json.

Also pins KC_HOSTNAME to a full public URL with KC_HOSTNAME_STRICT=true, required so the BFF's internal discovery fetch and the browser's real token issuance agree on the same issuer -- diverges intentionally from mcp-context-forge's own docker-compose.sso.yml convention, which doesn't need that consistency.

What changes

  • Add a keycloak service (quay.io/keycloak/keycloak:26.1) with the seeded realm. The app and gateway wait for it to be healthy, which adds up to about 30 seconds to every e2e run.
  • Extend e2e/seed/seed.ts to set trusted_for_api_auth and api_audience on the Keycloak provider.
  • Rotate the bootstrap admin password even when no forced password change applies. Previously that branch left the password unchanged.
  • Bump the gateway image from v1.0.9 to v1.0.11, matching openapi.json.
  • Set GUNICORN_WORKERS=1 and DEFAULT_USER_PASSWORD on the gateway, and make PORT configurable on the app.
  • Skip the mocked SSO suite under E2E_REAL_API and call mockPermissions() in its beforeEach.

Known limitation

The "Real Keycloak SSO login flow" tests are skipped in every mode. Gateway v1.0.11 does not dispatch external-IdP bearer tokens, and the fix ships in v1.0.12. Tracking issue: #190.

Notes

  • The realm is dev/test only. All secrets are fixed and committed.
  • Keycloak binds to 127.0.0.1:8180.
  • The contextforge-web-ui client allows callbacks on ports 3000 (HTTP) and 8443 (HTTPS). A custom PORT needs its own redirect URI.

Add a `keycloak` service and `e2e/keycloak/realm-export.json` to
docker-compose.e2e.yml so this repo's e2e suite never needs a sibling
mcp-context-forge checkout to exercise SSO. The realm seeds two clients
(`mcp-gateway` for the real gateway image, `contextforge-web-ui` for this
repo's own BFF) and one test user, modeled on (not synced with) mcp-context-forge's
own infra/keycloak/realm-export.json.

Also pins KC_HOSTNAME to a full public URL with KC_HOSTNAME_STRICT=true,
required so the BFF's internal discovery fetch and the browser's real
token issuance agree on the same issuer -- diverges intentionally from
mcp-context-forge's own docker-compose.sso.yml convention, which doesn't
need that consistency.
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>

Signed-off-by: Gabriel Costa <gabrielcg@proton.me>

@marekdano marekdano left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Functionally-impacting

1. Real Keycloak SSO tests skip in every mode

e2e/auth/sso-login.spec.ts:152-157

The describe carries both test.skip(!IS_REAL_API) and test.skip(IS_REAL_API), so neither test runs. Nothing in CI exercises the realm, the seed's trusted_for_api_auth step or the realSsoLogin() selectors. The PR delivers infra but no executing SSO coverage, yet it closes #6926.

Fix:

  • State in the PR description that the real tests stay disabled until gateway v1.0.12, and link an issue to remove the skip.
  • Run them once locally against a patched gateway to validate the realm and selectors.
  • Keep Closes only if #6926 is infra-only.

2. SSO_ENABLED default conflicts with .env.example

docker-compose.e2e.yml:16, .env.example

${SSO_ENABLED:-true} is read from .env during interpolation, and .env.example ships SSO_ENABLED=false. The TLS smoke workflow runs cp .env.example .env, so the app runs with SSO off while the gateway has it on. The "Sign in with Keycloak" button never appears.

Fix: Hardcode SSO_ENABLED: "true" in the e2e compose (as the gateway side does), or document that .env must set it.

3. Stale TLS comment in the seed

e2e/seed/seed.ts:128-130

The comment says the seed is a no-op in the TLS variant because its gateway has no provider. That variant stacks on docker-compose.e2e.yml, so its gateway now has the Keycloak provider and the seed runs fully.

Fix: Correct the comment. The code is fine.


🟡 Suggestion

4. Redirect URI fixed to port 3000 and HTTP

e2e/keycloak/realm-export.json, docker-compose.e2e.yml:14

redirectUris and webOrigins are fixed to http://localhost:3000, while PORT is configurable and the TLS variant serves on https://localhost:8443. It only bites when PORT is overridden or SSO is run under TLS, and the real tests are skipped. Upgrade to if the real tests are unskipped in this PR.

Fix: Add a wildcard URI (http://localhost:*/auth/sso/callback) or document the port 3000 and HTTP assumption.

5. Keycloak bound to all host interfaces

docker-compose.e2e.yml:93

"8180:8080" exposes Keycloak with a fixed admin password and fixed client secrets.

Fix: Use 127.0.0.1:8180:8080, and the same for the gateway port for consistency.

6. Floating Keycloak image tag

docker-compose.e2e.yml:79

quay.io/keycloak/keycloak:26.1 is a floating minor tag, so the e2e stack changes when the tag moves.

Fix: Pin to a patch version, ideally with a digest.

7. Keycloak startup cost on every suite

docker-compose.e2e.yml:24,74

The app and gateway both wait for Keycloak health (about 30s start period plus retries), including suites that never use SSO.

Fix: Accept the cost and note it in the PR description. A compose profile adds more complexity than it saves.

8. Weak error handling in the seed

e2e/seed/seed.ts (seedSsoTrustedApiAuth)

The provider loop silently continues on a failed detail fetch, so a transient 5xx later reads as "No SSO provider matches". The verify GET never checks verifyRes.ok.

Fix: Throw or log on a failed detail fetch. Check verifyRes.ok before .json().

9. Over-broad mcp-gateway client

e2e/keycloak/realm-export.json

directAccessGrantsEnabled and serviceAccountsEnabled are on but unused by the browser flow. The client has no audience mapper, so its tokens would fail the gateway's api_audience check.

Fix: Confirm intent. Otherwise disable the unused grants.


⚪ Minor

10. Redundant inner skips

e2e/auth/sso-login.spec.ts:81,122

The describe-level test.skip(IS_REAL_API) at line 61 makes the per-test skips redundant.

Fix: Remove the inner skips.

11. Transient process references

.env.example:107, e2e/auth/sso-login.spec.ts:145

"Task 6.2" and "mcp-context-forge#6928" appear in a comment and a test title. Comments must not carry transient process.

Fix: Describe the behavior and drop the task numbers.

12. Incomplete PR description

The description omits: the gateway bump v1.0.9 → v1.0.11 (matches openapi.json), GUNICORN_WORKERS=1, DEFAULT_USER_PASSWORD, PORT, the seed password-rotation change and the added apiMock.mockPermissions() in the mocked suite.

Fix: Add one line each.

marekdano and others added 2 commits October 13, 2026 16:02
  - Bind Keycloak to 127.0.0.1 so the fixed admin password and client
    secrets stay off other host interfaces.
  - Allow the HTTPS callback on port 8443 for the contextforge-web-ui
    client, and document that a custom PORT needs its own redirect URI.
  - Throw when the seed cannot read or re-read an SSO provider, instead
    of continuing silently.
  - Correct the seed comment about the TLS variant.
  - Remove redundant per-test skips from the mocked SSO suite. The
    describe-level skip covers them.
  - Link the skipped real SSO suite to #190.
  - Drop task numbers from comments and test titles.
  - Document in .env.example that the real SSO test needs
    SSO_ENABLED=true.

Signed-off-by: Marek Dano <mk.dano@gmail.com>
@marekdano

Copy link
Copy Markdown
Contributor

Pushed updates while Gabriel is away.

Summary

Changed

  • Keycloak now binds to 127.0.0.1:8180.
  • The contextforge-web-ui client allows the HTTPS callback on port 8443. The README notes that a custom PORT needs its own redirect URI.
  • The seed throws when it cannot read an SSO provider, instead of continuing silently.
  • The mocked SSO suite lost its redundant per-test skips. The describe-level skip covers them.
  • Task numbers are gone from comments and test titles.
  • .env.example documents that the real SSO test needs SSO_ENABLED=true.
  • The PR description now lists the omitted changes and the known limitation.

Not changed

  • The real SSO suite stays skipped until the gateway image reaches v1.0.12. Tracking issue: [CHORE]: Un-skip real Keycloak SSO e2e once gateway image is v1.0.12 or later #190. It is not validated against a live Keycloak yet.
  • SSO_ENABLED stays overridable from .env. The note in .env.example covers the default mismatch.
  • Keycloak stays on 26.1 to avoid manual patch bumps.
  • The mcp-gateway client keeps its original grants, matching the realm this one is modeled on.
  • The Keycloak startup cost stays. It adds up to about 30 seconds per e2e run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vendored e2e Keycloak service + realm export (self-contained, CI-safe)

2 participants