Skip to content

feat(express): add default domain-scoped CA trust for BitGo connections - #10073

Open
pranishnepal wants to merge 1 commit into
masterfrom
pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to
Open

pranishnepal wants to merge 1 commit into
masterfrom
pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to

Conversation

@pranishnepal

@pranishnepal pranishnepal commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What

Restricts outbound HTTPS connections from BitGo Express (and @bitgo/sdk-api when tlsPolicy is configured) to official BitGo domains (*.bitgo.com, *.bitgo-test.com, *.bitgo-staging.com, *.bitgo-dev.com, and their apex domains) to a bundled multi-provider Root CA store (modules/express/bitgo-ca.pem) by default, while keeping the standard Node.js / system trust store for all non-BitGo destinations (external signer, block explorers, webhooks, KMS).

  • Curated 17-root multi-provider CA bundle (modules/express/bitgo-ca.pem): Pre-includes the self-signed Root CAs for the 5 tier-1 providers BitGo uses today or would fail over to (Google Trust Services, Amazon Trust Services, DigiCert, Let's Encrypt / ISRG, and Sectigo). This shrinks the trusted CA surface for BitGo domains from ~150 global CAs down to 5 providers (covering app.*, admin.*, and cdn.bitgo.com out of the box) without breaking across routine leaf renewals, intermediate rotations, or CA migrations.
  • Single enforcement funnel (BitGoAPI.getAgentRequest): Every BitGo-bound request (requestPatch, fetchConstants, executeAssetRequest) goes through getAgentRequest, which attaches the restricted https.Agent({ ca }), refuses redirects (.redirects(0)) on covered hosts so a redirect cannot escape the domain policy, and refuses plaintext http:// downgrades on covered hosts.
  • Single boolean opt-out + proxy safety: Enabled by default (disablePinCerts: false). Operators can opt out with --disablepincerts or BITGO_DISABLE_PIN_CERTS=true. If BITGO_USE_PROXY is set, Express logs a startup warning and skips the bundled CA policy so existing proxy deployments continue to work.

Why this shape

NODE_EXTRA_CA_CERTS can only add trust; nothing in Node.js lets you remove a compromised CA from the global trust store, and replacing the global store would break non-BitGo outbound calls. Passing ca on a dedicated https.Agent replaces the trust store only for connections routed through that agent. Pairing appliesTo(url) with agent inside BitGoTlsPolicy guarantees that covered BitGo domains use the restricted bundle while all other destinations keep the system store.

What's in here

  1. @bitgo/sdk-api (modules/sdk-api/src/tlsPolicy.ts, bitgoAPI.ts, types.ts):
    • createBitGoTlsPolicy({ ca, hosts }) splits and validates every PEM certificate in ca via crypto.X509Certificate, normalizes and validates host patterns via parsePolicyHost, and returns { agent, appliesTo(url) }.
    • appliesTo(url) matches official BitGo hostnames (including trailing-dot FQDNs) and throws if a covered host is contacted over plain http:// or a schemeless URL.
    • getCaExpiries(ca) inspects certificate expiry dates for startup monitoring.
    • Enforced in BitGoAPI.getAgentRequest (bitgoAPI.ts:497-501) and wired into fetchConstants (bitgoAPI.ts:780).
  2. @bitgo/express (modules/express/src/tlsPolicy.ts, bitgo-ca.pem, config.ts, args.ts, expressApp.ts, clientRoutes.ts, fetchEncryptedPrivKeys.ts):
    • Ships modules/express/bitgo-ca.pem (17 self-signed Root CAs across GTS, Amazon, DigiCert, ISRG, and Sectigo).
    • Enables the policy by default across BITGO_DEFAULT_TLS_HOSTS (bitgo.com, .bitgo.com, bitgo-test.com, .bitgo-test.com, bitgo-staging.com, .bitgo-staging.com, bitgo-dev.com, .bitgo-dev.com).
    • Adds --disablepincerts / BITGO_DISABLE_PIN_CERTS (with proper boolean parsing so BITGO_DISABLE_PIN_CERTS=false stays enabled).
    • Validates the policy at boot in checkPreconditions (expressApp.ts:273) and warns via console.warn if any bundled CA expires within 30 days.
  3. CI & Live Sync Check (scripts/verify-bitgo-policy-data.sh, modules/express/package.json):
    • Runs automatically before express unit-test (yarn test:data), verifying that all 17 certificates in bitgo-ca.pem parse, are self-signed CA roots (issuer === subject and cert.ca === true), are >30 days from expiry, and cover all 13 BitGo apex/subdomain hosts.
    • Supports bash scripts/verify-bitgo-policy-data.sh --live to verify real TLS handshakes against live app.*, admin.*, and cdn.bitgo.com endpoints using only bitgo-ca.pem.

How it was tested

  • Unit tests: modules/sdk-api/test/unit/tlsPolicy.ts (18/18), modules/express/test/unit/tlsPolicy.ts, config.ts, bitgoExpress.ts (55/55), and full express unit suite.
  • CI & live endpoint verification (scripts/verify-bitgo-policy-data.sh --live): Verified live TLS handshakes against app.bitgo.com, app.bitgo-test.com, app.bitgo-staging.com, app.bitgo-dev.com, admin.bitgo*.com, and cdn.bitgo.com using only bitgo-ca.pem.
  • Real staging E2E (app.bitgo-staging.com): Verified authenticated SDK/Express calls (fetchConstants, me(), wallet read, session unlock) with the default 17-root policy enabled, plus negative rejection when an unapproved CA bundle is used (Linear test evidence).

TICKET: WCN-3731

@linear-code

linear-code Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

WCN-3731

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Compatibility checks are failing on Node.js 27-nightly (experimental, non-blocking). This does not block merge, but it signals an incompatibility to fix before this release becomes supported.

View run

@pranishnepal
pranishnepal marked this pull request as ready for review October 27, 2026 16:25
@pranishnepal
pranishnepal requested review from a team as code owners October 27, 2026 16:25

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@pranishnepal
pranishnepal marked this pull request as draft October 27, 2026 16:54
@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch 2 times, most recently from 7fc7465 to c8d0fe9 Compare October 28, 2026 05:03
@pranishnepal pranishnepal changed the title Pranishnepal/wcn 3731 add domain scoped ca trust and optional spki pinning to feat(express): add default domain-scoped CA trust for BitGo connections Oct 28, 2026
@pranishnepal
pranishnepal marked this pull request as ready for review October 28, 2026 05:17
@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch 2 times, most recently from d2625bb to 21198bd Compare October 28, 2026 17:48
Comment thread modules/express/package.json Outdated
"unprettied": "grep -R -L --include '*.ts' --include '*.js' --include '*.json' '@prettier' src test",
"fmt": "yarn prettier --write '{src,test}/**/*.{ts,js,json}'"
"fmt": "yarn prettier --write '{src,test}/**/*.{ts,js,json}'",
"test:data": "bash ../../scripts/verify-bitgo-policy-data.sh --live"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will run prior to the unit test and fail the CI if it needs updating ~30d prior to the actual expiration date

@bitgo/express: $ bash ../../scripts/verify-bitgo-policy-data.sh --live
@bitgo/express: ok: 18 self-signed CA certs valid (>30d), 13 BitGo hosts covered, http downgrades throw
@bitgo/express: live ok: https://app.bitgo.com/ (HTTP 302)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is expired, lets log what the releaser needs to do

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added as log under with RELEASER_STEPS in scripts/verify-bitgo-policy-data.sh

@zahin-mohammad

Copy link
Copy Markdown
Contributor

@claude review always

Comment thread modules/express/bitgo-ca.pem Outdated
@@ -0,0 +1,381 @@
# GlobalSign Root CA (validTo: Jan 28 12:00:00 2028 GMT)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Document how this file was generated and how to re-generate it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, added bash scripts/verify-bitgo-policy-data.sh --regenerate to do that!

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @zahin-mohammad's task in 54s —— View job


Review of PR #10073: default domain-scoped CA trust for BitGo connections

I read the changed source and the verification script. I did not run the tests or the build. git fetch was denied, so I couldn't compute the diff against origin/master. I read the files directly instead, and I didn't read the .pem or the unit tests.

The design is sound. The single funnel in getAgentRequest, the per-agent ca, and the appliesTo check that throws on http:// for covered hosts all fit together well. The findings below are mostly about operational risk.

Should address

  1. test:data makes live network calls and gates every express unit-test run (modules/express/package.json:38, scripts/verify-bitgo-policy-data.sh:119).

    • unit-test now runs yarn test:data && ..., and test:data passes --live. That does a real TLS handshake to app.bitgo.com.
    • Any unit-test run without outbound network to prod will fail: offline dev, sandboxed CI, or a prod blip.
    • The expiry check (<30d → exit 1) also fails every unrelated PR once a bundled root gets within 30 days of expiry. That is intended as an early warning, but it blocks unrelated work.
    • The script also requires sdk-api to be built first (dist/src/tlsPolicy.js).
    • Suggestion: keep the offline structural and expiry checks in unit-test, and move --live to a scheduled or separate CI job.
  2. redirects(0) may break legitimate flows (modules/sdk-api/src/bitgoAPI.ts:500).

    • With superagent, a 3xx response with redirects disabled is surfaced as an error, because ok() only accepts 2xx.
    • Your own live check shows app.bitgo.com/ returns HTTP 302.
    • executeAssetRequest goes through this path, and cdn.bitgo.com is a covered host. If any asset or API endpoint redirects, for example to a signed object-store URL, it will now fail.
    • Please confirm no BitGo endpoint relies on redirects. If one does, allow redirects only when the target is also covered, or let redirected hops use the default agent.
  3. new https.Agent({ ca }) doesn't enable keep-alive (modules/sdk-api/src/tlsPolicy.ts:136).

    • Node's global agent has keep-alive on by default (Node 19+), but a freshly constructed agent does not.
    • Every BitGo call from Express would then pay a full TCP and TLS handshake. That is a latency regression on a hot path.
    • Suggestion: pass keepAlive: true, and consider a maxSockets setting as well.

Worth considering

  1. Transparent TLS-intercepting proxies and NODE_EXTRA_CA_CERTS.

    • A custom ca on the agent ignores NODE_EXTRA_CA_CERTS.
    • Operators behind a corporate MITM proxy that doesn't use BITGO_USE_PROXY will get cert errors on upgrade, since the policy is on by default.
    • The README mentions the opt-out. Please also call out this specific upgrade failure mode and the error text, and consider noting it in release notes.
  2. Case-sensitive scheme check (modules/sdk-api/src/tlsPolicy.ts:153).

    • !url.startsWith('https://') rejects HTTPS://app.bitgo.com, which is valid, as an "insecure" connection.
    • It's unlikely in practice. parsed.protocol !== 'https:' already covers the real cases, and the schemeless fallback already forces http:, so the startsWith clause can probably be dropped.
  3. Doc mismatches in the script.

    • The header says --live checks the production endpoint, which matches the code. The PR description says it verified all of dev, staging, test and prod, and that only happens with --live-all.
    • The header also says --live-all checks "9 endpoints"; the list does have 9, so it's fine, but the CI gate only covers prod. A bundle that works for prod could still fail for staging or dev, so run --live-all before merging or on a schedule.
  4. Minor items.

    • The caBundlePath parameter of getBitGoTlsPolicy appears to be used only by tests and bypasses the cache. Consider documenting that.
    • The "disabled" and "proxy" warnings are emitted on every getBitGoTlsPolicy call (boot plus prepareBitGo). It's harmless, but it is two warnings per startup.
    • The Node 26.x unit-test failure reported by CI should be checked to confirm it isn't caused by this PR. The new crypto.X509Certificate and tls usage is a candidate. Only the runs/40739197895 link is available to me.

Verified OK

  • All three entry points (requestPatch, fetchConstants, executeAssetRequest) go through getAgentRequest.
  • tlsPolicy combined with customProxyAgent is rejected explicitly.
  • Express passes tlsPolicy only when no proxy is in use.
  • Host matching handles trailing dots, .domain subdomain patterns, and doesn't match look-alikes such as bitgo.com.evil.com.
  • bitgo-ca.pem is not excluded by .npmignore, and the Dockerfile copies all of modules/express.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🔴 modules/express/src/fetchEncryptedPrivKeys.ts — Running the express mocha suite with any CLI argument now kills the run with an argparse usage error, before any test executes. That includes a spec path, --grep or an IDE runner. The new module-level config() call parses process.argv through parser.parseArgs() when the file is imported. test/unit/clientRoutes/externalSign.ts imports this file, so mocha's own flags reach argparse and it exits with code 2. Fix: do not parse CLI args at import time. Build the policy from EnvConfig() merged with DefaultConfig, or create bg lazily inside fetchKeys/main, so importing the module has no argv side effects.

    Why this was flagged

    modules/express/src/fetchEncryptedPrivKeys.ts:44 now calls getBitGoTlsPolicy(config()) at module scope. config() (config.ts:200) calls args(), which is parser.parseArgs() over process.argv (args.ts, argparse ^1.0.10). That argparse version calls this.error('Unrecognized arguments ...') and then process.exit(2) on any unknown token. modules/express/test/unit/clientRoutes/externalSign.ts:66 imports fetchKeys from this module, and mocha's spec: ['test/unit/**/*.ts'] loads that file in the mocha process. So yarn mocha test/unit/clientRoutes/externalSign.ts, yarn mocha --grep foo, or an IDE mocha runner puts mocha's argv in front of argparse, and the process exits with BitGo-Express usage text. On the base branch the import has no argv side effect and these runs work. Plain yarn unit-test passes no argv, so CI still works.

    Verification: Trigger: someone runs the express mocha suite with any extra CLI token, such as yarn unit-test --grep foo. The module-scope call is at fetchEncryptedPrivKeys.ts:44. config() (config.ts:200-204) runs ArgConfig(args()). In argparse 1.x, leftover tokens lead to process.exit(2). Before this change nothing in src called args() at import time.

  • 🔴 modules/sdk-api/src/tlsPolicy.ts — After this change, every Express call to BitGo opens a new TCP and TLS connection, because pinning is on by default and the pinned agent does not reuse connections. new https.Agent({ ca: normalizedCa }) leaves keepAlive at false. On Node >=20 the global agent superagent used before keeps connections alive. Fix: build the agent with keepAlive: true, matching the global agent's reuse, so pinning does not add a full handshake to each BitGo request.

    Why this was flagged

    Express turns the policy on by default (disablePinCerts: false in config.ts) and passes it to every per-request BitGo client in prepareBitGo (clientRoutes.ts:1839). getAgentRequest (bitgoAPI.ts:500) then calls req.agent(this._tlsPolicy.agent) for every app.bitgo*.com URL. That agent is built at tlsPolicy.ts:136 with only ca, so keepAlive defaults to false and each socket closes after its response. On the base branch superagent uses https.globalAgent, which has had keepAlive on by default since Node 19, and the repo supports Node >=20. So every proxied wallet, sign or send call now pays an extra TCP and TLS handshake, which adds latency and CPU. Under load it can also cause TIME_WAIT and ephemeral port exhaustion. Nothing in the diff sets keepAlive or reuses a shared agent.

    Verification: The policy agent is built at modules/sdk-api/src/tlsPolicy.ts:136-138 as new https.Agent({ ca: normalizedCa }). Nothing sets keepAlive, so it defaults to false. getAgentRequest (bitgoAPI.ts:497-500) applies it to every covered URL. On the base branch superagent fell back to https.globalAgent, which has keepAlive on since Node 19. Every call from Express to BitGo now opens a new TCP connection and does a full TLS handshake.

Comment thread modules/express/package.json Outdated
Comment thread modules/express/src/clientRoutes.ts
Comment thread modules/sdk-api/src/bitgoAPI.ts Outdated
@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch from 21198bd to 04ef34b Compare October 30, 2026 03:06

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 other optional suggestions (nits or notes on pre-existing code) were found and not posted.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Still open from earlier reviews (1):

  • Unresolved: 1 minor or pre-existing.

Comment thread modules/sdk-api/src/tlsPolicy.ts
Comment thread modules/sdk-api/src/tlsPolicy.ts Outdated
@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch 2 times, most recently from 504ff59 to b6ff4f8 Compare October 30, 2026 03:19

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

The latest push brings back an issue flagged in an earlier review; replied in that comment's thread.

Still open from earlier reviews (2):

  • Unresolved: 2 minor or pre-existing.

@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch from b6ff4f8 to 53d5386 Compare October 30, 2026 03:33

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new blocking issues. 1 optional suggestion (a nit or a note on pre-existing code) was found and not posted. Nothing in this review needs a push before merging.

@pranishnepal
pranishnepal marked this pull request as draft October 30, 2026 03:52
@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch from 53d5386 to 59e67e8 Compare October 30, 2026 10:50
@pranishnepal
pranishnepal marked this pull request as ready for review October 30, 2026 11:13

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@pranishnepal

Copy link
Copy Markdown
Contributor Author

@claude review always

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @pranishnepal's task in 16s —— View job


Review of PR #10073 (latest push)

I read modules/sdk-api/src/tlsPolicy.ts on the current head. I did not run the build or tests, and I did not re-read the other changed files in this pass.

Addressed since earlier reviews

  • The pinned agent now sets keepAlive: true, timeout: 5000 and scheduling: 'lifo'. That matches Node's default global agent, so the ECONNRESET risk on idle pooled sockets is covered.
  • appliesTo now normalizes backslashes to / before the scheme check.
  • The scheme regex requires :/ after the scheme. That closes the host:port bypass.
  • The check is parsed.protocol !== 'https:', so uppercase HTTPS:// is accepted.

Still open from earlier threads (not re-verified here)

  • The test:data script runs --live as part of unit-test, so offline or sandboxed runs of the express suite fail. Consider moving --live to a separate or scheduled job.
  • redirects(0) turns any 3xx from a covered host into an error. The live check shows app.bitgo.com/ returns a 302. Please confirm no API or CDN endpoint redirects.
  • fetchEncryptedPrivKeys.ts calls config() at module scope, which parses process.argv. Running mocha with extra args, such as --grep, can then exit with an argparse error. Create the policy lazily instead.
  • A TLS-inspecting egress that trusts its CA through NODE_EXTRA_CA_CERTS will fail on BitGo calls. A startup warning that names --disablepincerts would help operators.

No new issues in the reviewed file.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread modules/sdk-api/src/tlsPolicy.ts
Comment thread modules/sdk-api/src/tlsPolicy.ts
Comment thread modules/sdk-api/src/bitgoAPI.ts
Comment thread modules/express/test/unit/tlsPolicy.ts
Comment thread scripts/verify-bitgo-policy-data.sh Outdated
Restrict outbound HTTPS connections to official BitGo domains
(*.bitgo.com, *.bitgo-test.com, *.bitgo-staging.com, *.bitgo-dev.com,
and their apex domains) to a bundled multi-provider root CA store
(Google Trust Services, Amazon Trust Services, DigiCert, Let's
Encrypt/ISRG, and Sectigo) while keeping the system trust store for all
non-BitGo destinations:
- sdk-api: add createBitGoTlsPolicy({ ca, hosts }) and enforce it in
  BitGoAPI.getAgentRequest (refusing redirects and plain http on covered
  hosts)
- express: enable the bundled 18-root CA policy by default at startup,
  with a single opt-out flag (--disablepincerts /
  BITGO_DISABLE_PIN_CERTS)
  and automatic proxy fallback when BITGO_USE_PROXY is set
- scripts/verify-bitgo-policy-data.sh: CI check verifying all bundled
  roots are self-signed, valid (>30d), and cover all BitGo domains
  (plus --live handshake verification)

TICKET: WCN-3731
@pranishnepal
pranishnepal force-pushed the pranishnepal/wcn-3731-add-domain-scoped-ca-trust-and-optional-spki-pinning-to branch from 59e67e8 to 47e5545 Compare October 30, 2026 12:00

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants