Skip to content

Add Graphile readiness and generation-scoped disposal - #1747

Open
Zetazzz wants to merge 3 commits into
mainfrom
agent/graphile-readiness-disposal
Open

Zetazzz wants to merge 3 commits into
mainfrom
agent/graphile-readiness-disposal

Conversation

@Zetazzz

@Zetazzz Zetazzz commented Aug 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem and result

Evicting a Graphile instance on main releases PostGraphile but leaves its preset PostgreSQL services alive. An active subscription therefore keeps a database connection checked out and continues receiving notifications after its instance is evicted.

This PR releases generation-owned services during disposal and failed-build cleanup, and makes cleanup failures observable. The reproduced connection leak is eliminated: four successive evictions leave zero checked-out connections instead of four.

Before / after verification

Compared a fresh build of main 299b3920 with the rebased implementation 241db5e1, using Node 24.20.0, PostgreSQL 18.6, and unmodified @dataplan/pg 1.1.1. The final commit 209fa6b9 changes only CI test placement.

Each of four sequential cycles creates a real Graphile instance with a fresh service, executes a GraphQL read, confirms subscription readiness through notification delivery, and evicts the instance. All cycles share one caller-owned pool. Observations are recorded before fixture teardown or explicit cleanup, so test teardown cannot hide leaked connections.

Check main This PR
Connections still checked out after successive evictions 1 → 2 → 3 → 4 0 → 0 → 0 → 0
Service release calls per generation 0 Exactly 1
Retired subscription Continues receiving notifications Iterator terminates
Identical cleanup assertions Fail Pass
Caller-owned pool remains usable Yes Yes

All eight GraphQL reads returned the expected value, and schema hashes matched across both versions. With an injected schema-build failure and an already active subscriber, the PR also returned the checked-out connection (1 → 0), released the service exactly once, preserved the original error, and produced no unhandled rejection.

Test coverage

  • 128 local tests passed: graphile-cache, 6 suites / 28 tests; perf-harness, 11 suites / 100 tests.
  • 72 real PostgreSQL lifecycles passed across 7 cases: idle services, active subscriptions, subscription startup/release races, Graphile disposal, subscribed Graphile disposal, failed builds, and same-key generation replacement. Every case reached zero checked-out connections, queued acquisitions, and LISTEN channels; database backends returned to the two-fixture-client baseline before teardown. This was a functional run with one repetition, concurrency 1, and a two-connection pool.
  • All 17 CI jobs passed on 209fa6b9: CI run 41773958164, including Windows and PostgreSQL tests. The complete graphile-cache suite runs in the PostgreSQL batch; CI matrix coverage still includes all 118 packages with tests.
  • CJS/ESM package builds, frozen offline installation, dependency policy, and diff checks passed. Focused lint has no errors; the existing unused unregister warning remains.

The rebase preserves main's heap measurement, cache budget, and LRU behavior. Regression coverage verifies that budget rejection and subsequent disposal release PostGraphile and its services only once. Cleanup attempts the remaining resources after a failure, and pool shutdown is attempted even when generation cleanup rejects.

Conclusion and limits

This is a demonstrated resource-lifecycle fix, with practical value for subscription workloads that repeatedly rebuild or evict Graphile instances. It prevents retired generations from accumulating checked-out connections and consuming pool capacity. Query throughput and latency improvements were not measured; no speedup percentage is claimed.

  • Each tested generation owns a fresh service object. Sharing one service object across live generations is not addressed.
  • Public service release can finish before upstream background UNLISTEN/client return. Tests poll actual pool quiescence separately. Startup/release churn briefly queued 18 acquisitions and settled in about 82 ms in this run.
  • Upstream event-listener retention remains unresolved: 20 subscription cycles left up to 10 notification and 10 error listeners per reused client, with two listener-limit warnings. This PR does not patch the dependency or claim to eliminate every lifecycle leak.
  • Large-scale and long-duration load behavior was not measured.

Reproduce the lifecycle suite

Build graphile-cache and @constructive-io/perf-harness, then run against a local PostgreSQL test administrator account. The suite creates and tears down isolated databases through pgsql-test.

node packages/perf-harness/dist/connection-lifecycle/suite.js \
  --database-url '<local PostgreSQL test administrator URL>' \
  --output /tmp/graphile-connection-lifecycle.json

Refs constructive-io/constructive-planning#1693 (F04).

@Zetazzz
Zetazzz force-pushed the agent/graphile-readiness-disposal branch from dec013c to e77d653 Compare September 6, 2026 01:51
@Zetazzz
Zetazzz force-pushed the test/performance-harness-core branch 2 times, most recently from 658736e to f7c63ec Compare September 7, 2026 03:22
@Zetazzz
Zetazzz force-pushed the agent/graphile-readiness-disposal branch from e77d653 to e7d7636 Compare September 9, 2026 04:19
@Zetazzz
Zetazzz changed the base branch from test/performance-harness-core to main September 9, 2026 04:19
@Zetazzz Zetazzz changed the title Add Graphile build readiness and awaited disposal Add Graphile readiness and generation-scoped disposal Sep 21, 2026
@Zetazzz
Zetazzz marked this pull request as ready for review September 21, 2026 15:47
@tenki-reviewer

tenki-reviewer Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Review complete. 🟡 1 medium

💬 Inline comments (1)

  • 🟡 Replacement/failed-build modes skip real release accounting — worker.ts:200

This PR reworks graphile/graphile-cache service lifecycle semantics.

Files Change
graphile-cache/src/{preset-services,create-instance,build-readiness,graphile-cache,index}.ts Introduces preset service management, a generation-replacement disposal model, build-readiness gating, and cache-wide disposal/close helpers.
graphile-cache/src/__tests__/* Adds lifecycle, disposal, build-readiness, and service-release unit tests using the pgsql-test harness.
perf-harness/src/connection-lifecycle/{suite,worker}.ts New load suite exercising concurrent cache-generations with leak/channel detection and latency metering.
graphile-cache/jest.config.js, package.json, .github/workflows/run-tests.yaml Wires the new package into CI with a ts-jest transform and workspace dev-dependencies.

The change is well-structured and well-tested; the main concern is that the perf-harness metering in replacement/failed-build modes reports placeholder release accounting rather than real disposals, which caps the harness's ability to detect leaks on those paths.

Reviewed commit: 1c9dfb1

@tenki-reviewer tenki-reviewer Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adds caching-instance and service-lifecycle logic to graphile-cache (preset services, build readiness, disposal) plus a perf-harness connection-lifecycle suite, with new CI wiring.

Key findings

  • 🟡 Replacement/failed-build modes skip real release accounting — worker.ts:200

Consolidate PR #1747, including its merge-owned release instrumentation, before rebasing onto current main.
@Zetazzz
Zetazzz force-pushed the agent/graphile-readiness-disposal branch from 89f4561 to 241db5e Compare November 4, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant