Repository navigation
feat: PostgreSQL integration tests and the work queue core (#277, #278) - #289
Merged
Merged
Conversation
Adds Anichron.Tests.Integration, which starts one postgres:17-alpine container per run and gives each test its own database, migrated or empty. First callers: EfRefreshTokenRepository.DeleteExpiredAsync and MigrateWithAdvisoryLockAsync (two concurrent migrators, lock released, lock held elsewhere), both PostgreSQL-only and untested until now. Test projects are classified by name (*.Tests.Unit, *.Tests.Integration) and a TestSuite property (Unit by default, Integration, All) selects which are test projects. Directory.Build.targets sets IsTestProject and, for projects outside the suite, IsTestingPlatformApplication=false: the test SDK and xUnit v3 set both to true after Directory.Build.props, and Stryker treats IsTestingPlatformApplication=true as a test project regardless of IsTestProject. Verified: Stryker's scoped run finds 1297 tests (unit only) instead of 1303. CI gets an "Integration tests" job, added to the gate's needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pooled check connection could reuse the migrator's own session, where an advisory lock is re-entrant, so the lock-released test could pass with the release broken. Shown failing with ReleaseAsync made a no-op. CLAUDE.md: a project or filter run of integration tests needs -p:TestSuite=Integration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The durable work queue's data layer (#276): - WorkItems, with the claim, queued-dedup, lease and UpdatedAt indexes, table and column comments, and its migration. - IWorkPayload (static abstract kind, opt-in dedup key), IWorkHandler, IWorkProgress and WorkJson (System.Text.Json with NodaTime) in Core. - EfWorkQueue, the only insert path. A queued item with the same key is returned instead of a second; a lost race on the partial unique index is absorbed and looked up again. - EfWorkItemStore in the Worker: claim over FOR UPDATE SKIP LOCKED, heartbeat, succeed, fail and release fenced by (Id, Attempt), lease recovery with the stall limit, due-ness and the retention delete. StorageConfigId and CancelRequestedAt land with #281 and #280. Contract tests on real PostgreSQL cover concurrent claims, skip-locked, zombie attempts, the stall limit, release without a stall and dedup coalescing. Each was shown failing with its guard removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…up on the queue (#278) - WorkHandlerRegistry dispatches by kind through registrations built in DI, never a switch, and fails startup unless every WorkKind has exactly one payload type in Core and exactly one handler, for that payload. - WorkExecutor claims one item at a time (oldest first) and runs it under a lease, heartbeating every 2 s with the progress snapshot. A fenced heartbeat cancels the handler and writes nothing more; a shutdown releases the item without a stall; a handler exception, including a cancellation that is neither, fails it with its error. - WorkScheduler recovers expired leases and enqueues each periodic kind that is due, from the table: nothing queued or running with its key and nothing finished within its interval. - TokenCleanup and WorkItemRetention as kinds with handlers and schedules. TokenCleanupService and its timer are gone. - Settings: poll 5 s, heartbeat 2 s, lease 60 s (must exceed the heartbeat), 3 stalls, retention 30 days, retention purge 24 h. - The Worker registers AnichronDbContext through a context factory, so the store can open a short-lived context per call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ADR 0008: a hand-rolled PostgreSQL work queue over a job library, with the evaluation from #273. - Architecture rules only-work-queue-inserts-work-items (only EfWorkQueue constructs a WorkItem) and work-handlers-live-in-worker. Shown failing on deliberate violations: rule 'only-work-queue-inserts-work-items' ... {"Anichron.Worker.ViolationInsert"} rule 'work-handlers-live-in-worker' ... {"Anichron.API.ViolationHandler"} - CLAUDE.md describes the work queue; the Worker no longer has a token cleanup timer. The RefreshTokens table comment follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#278) Review findings on the store: - Enqueue coalesces only queued items, so an item can be queued while another with its key runs. Releasing or recovering the running one then hit the queued-dedup unique index; recovery was one bulk update, so a single collision stopped all lease recovery and scheduling. Items are now handed back one at a time: requeued when no item with the key is queued, otherwise Cancelled as superseded, since the queued one does the same work. A racing insert is retried. Recovery re-checks the lease per item, so it never requeues one whose owner just beat. - The claim parsed the kind after claiming the row, so a kind from a newer release was left Running until it failed as stalled. The claim now takes only the kinds this Worker knows; others stay queued. Each new contract test fails against the previous store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #277. Closes #278. Part of #276; #279, #280 and #281 build on it.
What changes
PostgreSQL integration tests (#277). A new
Anichron.Tests.Integrationproject runs onepostgres:17-alpinecontainer per test run through Testcontainers, and gives each test its own database, either empty or with the real migrations applied. A test keeps the image in step withsrc/docker-compose.yml. Its first callers are code that only real PostgreSQL can exercise and that had no tests:DeleteExpiredAsync;MigrateWithAdvisoryLockAsync: two concurrent migrators both finish, the lock is released afterwards, and with the lock held elsewhere the migrator gives up without migrating.Test suites by name.
Directory.Build.propsclassifies projects ending in.Tests.Unitand.Tests.Integration. ATestSuiteproperty (Unitby default,IntegrationorAll) decides which of them are test projects. Plaindotnet testand Stryker therefore never run the integration suite.Directory.Build.targetssetsIsTestProjectfor projects in the selected suite.IsTestingPlatformApplication=false. Microsoft.NET.Test.Sdk and xUnit v3 set both properties to true afterDirectory.Build.props, and Stryker treatsIsTestingPlatformApplication=trueas a test project whateverIsTestProjectsays. Before this fix the scoped Stryker run found 1,303 tests, the 6 integration tests included; now it finds only the unit tests.Integration tests, runsdotnet test src/ -p:TestSuite=Integrationand is added to the gate'sneeds.Work queue core (#278). A hand-rolled, durable queue in
WorkItems(ADR 0008 records why not a library, citing #273). Refresh-token cleanup is its first caller.IWorkQueue.EnqueueAsync(EfWorkQueue, Core) is the only insert. A payload with aDedupKeycoalesces: while an item with that key isQueued, enqueueing again returns that item. A partial unique index absorbs racing inserts.EfWorkItemStore(Worker).FOR UPDATE SKIP LOCKED, and incrementsAttempt.(Id, Attempt), so an attempt that lost its lease changes nothing.Cancelledas superseded.TokenCleanupreplacesTokenCleanupServiceand its timer.WorkItemRetentiondeletes items that finished more than 30 days ago.WorkHandlerRegistrydispatches by kind through registrations made in DI, never aswitch. It fails startup unless everyWorkKindhas exactly one payload type in Core and exactly one handler for it.Worker:WorkPollIntervalSeconds,WorkHeartbeatSeconds,WorkLeaseSeconds(must be longer than the heartbeat),WorkMaxStalls,WorkItemRetentionDaysandWorkItemRetentionIntervalHours, with spec: Work queue — durable API/Worker work items with scope routing, leases and a live admin view #276's defaults.Decisions made with the maintainer
StorageConfigIdandCancelRequestedAtare not in this migration. They arrive with their first callers, feat(worker): Crawl as a work item per storage config, with scope routing #281 and feat: Cancel queued and running work items #280.Cancelled, superseded). spec: Work queue — durable API/Worker work items with scope routing, leases and a live admin view #276's rule that only queued items coalesce stays.Deliberately not here
Kindthis build does not know. Nothing here loads aWorkItem(the claim reads raw columns and skips unknown kinds), but the list and detail endpoints will.RevokeAllActiveByUserIdAsyncandExecuteInTransactionAsyncare PostgreSQL-only too, but still have no integration tests; test: PostgreSQL integration test project (Testcontainers) for PostgreSQL-only code #277 did not ask for them.Verification
dotnet test -p:TestSuite=Allafter merging master: 1,416 unit and 31 integration tests, all passing.Attemptfence removed;SKIP LOCKEDremoved (the claim then blocks until the command timeout);<changed to<=in the expired-token delete;scripts/mutation-changed.shrun over every changed production file reached 100.00%. The PostgreSQL-only members carry// Stryker disable all, with ADR 0005 rows.mutation-guard.sh,derive-db-docs.sh --checkanddotnet format --verify-no-changespass.🤖 Generated with Claude Code