Skip to content

fix(rate-limiter): exclude prompts from per-server tool quotas - #203

Merged
lucarlig merged 1 commit into
mainfrom
fix/rate-limiter-prompt-quotas
Oct 14, 2026
Merged

lucarlig merged 1 commit into
mainfrom
fix/rate-limiter-prompt-quotas

Conversation

@vishu-bh

Copy link
Copy Markdown
Collaborator

Enabling by_user_per_server currently rejects ordinary prompt fetches without server_id with RATE_LIMIT_CONTEXT_MISSING (503). This change confines catalog-server quotas to tool invocations and restores ordinary prompt retrieval.

Prompt evaluation uses a Rust engine configuration view with by_user_per_server disabled, sharing the existing backend and clock with tool evaluation. User, tenant, and tool/prompt-name counters remain shared across hooks. Tool identity validation, counter keys, algorithms, failure policies, and Python entry points are unchanged.

Adds memory and live Redis regression coverage for prompt quota exclusion and shared counters, updates hook-specific documentation, and bumps the package to 0.1.12 with matching Cargo lockfile and manifest versions.

Closes #202.

Validation

  • make plugins-validate: passed (41 repository tests).
  • Plugin make ci test-all test-integration: passed; 92 Rust tests and 134 Python hook tests, including live Redis and TLS, with no skips. Formatting, Clippy with -D warnings, release-wheel build/install, and benchmark compilation passed.
  • Reproduction against installed CPEX: ordinary prompt without server_id succeeds; prompt and tool calls still share the global user quota.
  • git diff --check: passed.

Local validation used nextest 0.9.133. The existing TLS fixture generates a certificate missing an Authority Key Identifier, which Python 3.13 rejects before calling the plugin. The final run used the fixture's supported RATE_LIMITER_TLS_REDIS_URL override with an isolated Redis endpoint and a valid temporary certificate; fixture code and certificate verification were unchanged. One existing one-second Redis TTL test failed once and passed on the full rerun.

Gateway REST GET/POST and MCP prompts/get acceptance tests remain pending in the gateway repository. This PR does not publish a release or update gateway dependencies.

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>

@gandhipratik203 gandhipratik203 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lucarlig lucarlig left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lucarlig
lucarlig merged commit 1b4577f into main Oct 14, 2026
29 checks passed
@lucarlig
lucarlig deleted the fix/rate-limiter-prompt-quotas branch October 14, 2026 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: cpex-rate-limiter 0.1.11 rejects ordinary prompts when by_user_per_server is enabled

3 participants