[texastony]

Issue #, if available:

Description of changes:

boto3 binds AWS credentials to a client when the client is constructed, and every boto3 client (and Session) owns its own connection pool. So an application that must call KMS with per-tenant credentials needs a Session and a KMS client per tenant. Measured at ~239 ms and ~8.4 MiB each, that does not scale to hundreds or thousands of tenants. boto3 has no public way to override credentials per request (unlike the AWS SDK for Java's overrideConfiguration), and a search of boto3 and botocore issues found no feature request for one.

This adds examples that share one connection pool across every tenant. examples/src/multi_tenant/design.md records the options considered and the decisions made.

Shared building blocks (examples/src/multi_tenant/; Python 3.8+, no MPL import):

• kms_transport.py: KmsTransport, one per process. It owns:

  • one urllib3 pool
  • SigV4 signing via botocore.auth.SigV4Auth
  • basic retries modeled on botocore's standard mode
  • the same exception types boto3 raises (the mapping mirrors botocore/httpsession.py)

  It rejects any region that could change the endpoint host, because Discovery keyrings and providers take the region from the ciphertext. HTTP(S) proxies are out of scope.

• tenant_kms_client.py: TenantKmsClient, one per tenant per region. A boto3-shaped client for GenerateDataKey, GenerateDataKeyWithoutPlaintext, Encrypt, Decrypt, and ReEncrypt. It holds only a reference to the transport, a region, and the tenant's credentials.

Keyring examples (MPL):

• multi_tenant_kms_keyring_example.py: an AWS KMS Keyring per tenant, plus a TenantClientSupplier so an AWS KMS MRK Discovery Multi-Keyring decrypts with the tenant's credentials.
• multi_tenant_hierarchical_keyring_example.py: an AWS KMS Key Store and Hierarchical Keyring per tenant, sharing one DynamoDB client and one cache:
  • Shared storm tracking cache: every tenant's keyring uses one CacheTypeShared cache, so the fan out limit (thundering herd mitigation) applies across tenants. Every StormTrackingCache parameter is set explicitly, and the source explains each one. fan_out is raised to 100 and in_flight_ttl lowered to 5 s, because a cache shared by every tenant makes both limits process-wide (see design.md 10.1.1). These values are illustrative and should be tuned to the application's usage.
  • Partition ID: the SHA-384 digest of the tenant's secret access key, so tenants are also isolated within the cache without the keyring holding a copy of the secret. When credentials are refreshed, the Partition ID changes, so the first lookup after a refresh misses the cache. This is intended.
  • Requires MPL 1.11.3 or later: earlier Python releases of the storm tracking cache sleep sleep_milli seconds instead of milliseconds (Python StormTrackingCMC follower wait sleeps in whole seconds, not milliseconds aws-cryptographic-material-providers-library#1940). This is documented, not enforced; the ESDK's MPL extra still allows >=1.7.4.

Master Key Provider examples (examples/src/legacy/multi_tenant_kms_master_key_provider.py):

• Tenant variants of the Strict, MRK-aware Strict, Discovery, and MRK-aware Discovery KMS Master Key Providers, whose regional clients are TenantKmsClients.
• MRKAwareDiscoveryAwsKmsMasterKeyProvider._new_master_key_impl constructs MRKAwareKMSMasterKeyConfig directly instead of using master_key_config_class, so its subclass overrides that method (see design.md 4.2.2).

Branch Key cleanup in tests (examples/test/branch_key_cleanup.py):

• The Hierarchical Keyring examples create new Branch Keys on every run. delete_branch_key is a port of DeleteBranchKey from the MPL's Dafny test helpers (AwsCryptographyKeyStore/test/CleanupItems.dfy), and cleanup_branch_keys wraps it as best effort: it logs failures and never fails the test.
• The source states that Branch Keys SHOULD NOT be deleted in production, because deleting one cryptographically shreds every message it protected.

Changes to existing files:

• examples/src/hierarchical_keyring_example.py:
  • takes an optional branch_key_ids dict, so its test can clean up its Branch Keys (the test previously left two behind per run)
  • its docstring now gives the Key Store table's actual key schema: branch-key-id / type, not partition_key / sort_key
• examples/test/test_i_hierarchical_keyring_example.py: cleans up those Branch Keys in a finally.

Testing

• Offline: an in-memory KMS behind the transport's pool authorizes each KMS key for exactly one access key. The tests:

  • run the examples themselves
  • cover every keyring and MKP variant and every KMS operation above
  • assert which tenant signed every request, including that one tenant's credentials cannot use another tenant's key

  The Partition ID and the Branch Key cleanup also have offline tests (the latter use botocore's Stubber). 63 pass.

• Against AWS KMS (local): the AWS KMS Keyring and MKP examples pass. Requests signed with two different sets of credentials shared one TLS connection.

• Not yet verified against AWS: GenerateDataKeyWithoutPlaintext, ReEncrypt, and the Hierarchical Keyring examples. The local test identity lacks KMS permissions on the hierarchical test key, and fails the same way with the existing hierarchical_keyring_example.py. The Branch Key cleanup needs dynamodb:Query and dynamodb:DeleteItem on KeyStoreDdbTable.

• Static analysis: passes with the repo's pinned versions, including flake8-examples, pylint-examples, and isort-check (isort 5.11.4).

• CI on the author's fork: the CodeBuild jobs fail because the fork has no CI_AWS_ROLE_ARN. Separately, the decrypt-oracle jobs and the Python 3.8/3.9 tests jobs fail in actions/setup-python, because those versions are not available on the Ubuntu 26.04 runner image. That is unrelated to this change.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Check any applicable:

• Were any files moved? Moving files changes their URL, which breaks all hyperlinks to the files.

🤖 Generated with Claude Code