Skip to content

improvement(trigger): downsize connector sync and document processing machines - #8699

Merged
waleedlatif1 merged 1 commit into
stagingfrom
improvement/trigger-machine-downsize
Oct 6, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
improvement/trigger-machine-downsize

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • knowledge-connector-sync: large-1x → medium-2x. Runs are I/O-bound (source APIs + rate-limit backoff, near-zero CPU) and p99.9 RSS is ~650 MB; 4 GB still clears the 2.6 GB worst case. Not medium-1x because the task intentionally has no outOfMemory escalation (an OOM strands the sync lock until the reaper), so the preset has to hold the peak on its own
  • knowledge-process-document: medium-2x → medium-1x. p99.9 RSS ~740 MB, peak just under 2 GB; the existing outOfMemory: large-2x escalation catches the rare document past 2 GB
  • Trigger.dev bills wall-clock per preset, so this halves the per-second rate on both, and these two tasks are the largest share of background compute spend
  • Checked the last 30 days for OOM kills: none on either task at their current presets, and no dispatch site overrides machine at trigger time

Type of Change

  • Improvement

Testing

  • bun run --cwd apps/sim test background/knowledge-processing.test.ts background/knowledge-connector-sync.test.ts (42 passed)
  • Root bun run test, bun run lint, bun run check:audits, bun run type-check, bun run docs-manifest:check, block-registry check — all green
  • Sizing taken from Trigger.dev per-run peak process.memory.usage and CPU metrics over 30 days; will watch for TASK_PROCESS_OOM_KILLED after deploy

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Oct 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 6, 2026 10:33pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Downsizes background job machine presets for two tasks.

The PR appears safe to merge; no actionable failures were found.

What we checked:

  • Document retries can reclaim work: The worker passes the same queue token to processDocumentAsync. Its claim accepts processing rows with that token, so the unfinished status alone does not block the retry.

Summary

This PR lowers the default machine sizes for two knowledge tasks.

  • knowledge-connector-sync changes from large-1x to medium-2x.
  • knowledge-process-document changes from medium-2x to medium-1x, keeping its large-2x out-of-memory retry.
  • The comments explain the sizing choices using production measurements. No actionable issues were found. No runtime tests were run during this review.

Reviews (1) · Last reviewed commit: "improvement(trigger): downsize connector..."

@waleedlatif1
waleedlatif1 merged commit d59e923 into staging Oct 6, 2026
24 checks passed
@waleedlatif1
waleedlatif1 deleted the improvement/trigger-machine-downsize branch October 6, 2026 22:39

This branch was previously deployed

1 inactive deployment
Preview — 4c022e5d Deployed Oct 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant