Skip to content

Add queue benchmarks and record results - #235

Open
ianmacartney wants to merge 9 commits into
ian/shared-start-documentsfrom
ian/queue-benchmarks
Open

ianmacartney wants to merge 9 commits into
ian/shared-start-documentsfrom
ian/queue-benchmarks

Conversation

@ianmacartney

@ianmacartney ianmacartney commented Sep 9, 2026 •

Copy link
Copy Markdown
Member

Add queue latency/scheduling probes and reproducible benchmarks for baseline, timestamp ordering, and packed pending starts. Record task outcomes and package versions, remove unused probes, and fix cleanup skipping tied creation timestamps. Validate run settings, logging, and enqueue coverage before accepting results.

September 9 median paired throughput changes:

Change Mutations Actions
Timestamp ordering +14.3% +13.6%
Packing pending starts +4.7% +7.3%

Packing reduced enqueue writes by 49.5%. Its mutation throughput gain was variable (-4.9% to +10.3%). Full results and methodology include the cleanup interruption and all measured revisions.

Validation: 183,000 successful benchmark tasks; 175 workpool tests and 15 benchmark-tooling tests pass; both run in CI. Build, typecheck, lint, dashboard build, formatting, and Python checks pass.

ianmacartney commented Sep 9, 2026 •

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@ianmacartney ianmacartney changed the title test(benchmarks): add queue latency and scheduling probes Add queue benchmarks and record results Sep 9, 2026
@pkg-pr-new

pkg-pr-new Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/convex/workpool/@convex-dev/workpool@235

commit: 5352d16

@ianmacartney
ianmacartney force-pushed the ian/queue-benchmarks branch 2 times, most recently from 0fa5206 to 420c1cf Compare September 10, 2026 00:05

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Not reviewed (too large): example/benchmarks/2026-09-08.json (~5,865 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread package.json
Comment thread example/convex/test/cleanup.ts Outdated
Comment thread example/convex/test/latency.ts Outdated
Comment thread example/convex/test/latency.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files (changes from recent commits).

Not reviewed (too large): example/benchmarks/2026-09-09/results.json (~7,332 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread example/benchmarks/2026-09-09/analyze.py
Comment thread example/benchmarks/2026-09-09/run.py
Comment thread example/benchmarks/2026-09-09/run.py Outdated
Comment thread example/benchmarks/2026-09-09/prepare.py
Comment thread example/benchmarks/2026-09-09/prepare.py Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread example/benchmarks/2026-09-09/run.py
@ianmacartney
ianmacartney force-pushed the ian/shared-start-documents branch from 387a3ae to a9d1527 Compare September 10, 2026 06:21
@ianmacartney
ianmacartney force-pushed the ian/shared-start-documents branch from a9d1527 to 24b91d6 Compare September 11, 2026 20:19
@ianmacartney
ianmacartney force-pushed the ian/shared-start-documents branch from 24b91d6 to 95b2ccd Compare September 12, 2026 00:02
@ianmacartney
ianmacartney force-pushed the ian/queue-benchmarks branch 2 times, most recently from f85cd47 to e8216bb Compare September 15, 2026 23:31
@ianmacartney
ianmacartney force-pushed the ian/shared-start-documents branch from 95b2ccd to 5b038c1 Compare September 15, 2026 23:31
@ianmacartney
ianmacartney force-pushed the ian/shared-start-documents branch from 5b038c1 to a506b21 Compare October 9, 2026 06:24
@ianmacartney
ianmacartney force-pushed the ian/shared-start-documents branch from a506b21 to 215149a Compare October 10, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant