Repository navigation
fix(celery): capture task exceptions with exception autocapture - #1072
posthog[bot] wants to merge 1 commit into
Conversation
posthog-python Compliance ReportDate: 2026-11-03T10:24:30.254305+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the |
Celery catches task exceptions, so sys.excepthook never sees them. Exception autocapture now connects to the Celery task_failure and task_retry signals when Celery is imported. An instrumented PosthogCeleryIntegration replaces this capture, so each exception is captured once. The integration also captures the exception of a retried task. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 40cf8650-18a1-49a4-8f20-dff6cb63b0c8
246f54a to
d6b399a
Compare
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the |
|
Closing this approach for now. Automatically attaching Celery signal handlers only when Celery happens to be imported before client creation makes exception capture depend on initialization order. The process-global integration registry also makes deduplication and task-filter behavior ambiguous with multiple clients, while reporting each retry exception as a warning is a separate behavior change. I would rather use a single, clearly owned Celery instrumentation path with explicit exceptions-only vs. lifecycle configuration, or keep Celery capture opt-in and document it. Thanks for the investigation and tests! |
💡 Motivation and Context
enable_exception_autocaptureget no$exceptionevents for Celery task errors. They must find and callPosthogCeleryIntegration.instrument()themselves, and the error tracking docs do not mention it.sys.excepthookandthreading.excepthook. Celery catches every task exception in its trace code, so those hooks never run.PosthogCeleryIntegration._on_task_retrydid not capture the exception of a retried attempt.Changes
sys.modulesat client creation,ExceptionCaptureconnects totask_failureandtask_retry. A Celery worker always imports Celery first, so workers are covered. Processes that do not use Celery import nothing new.PosthogCeleryIntegrationwithcapture_exceptions=Trueis instrumented, autocapture skips the exception. The integration captures it once and applies itstask_filter.Retry.exc. A retry with no exception captures nothing.ExceptionCapture.close()disconnects the signals. Lifecycle events and context propagation stay opt-in through the integration.sourcelevelmechanism.handledtask_failurecelery.task_failureerrorfalsetask_retrycelery.task_retrywarningtrueNote
Behavior change: projects with autocapture and Celery will start to get
$exceptionevents for task errors (and per-attempt retry warnings). The rate limiter of autocapture applies to these events too. Count$exceptionevents with mechanism sourcecelery.task_failure/celery.task_retryto measure adoption.Follow-up outside this repo: add a Celery section to the Python error tracking install docs.
💚 How did you test it?
close()disconnects, no Celery) and for retry capture in the integration.uv run --with celery), with a task that raisesKeyErrorand a task that retries once:main$exceptioneventsKeyErrorerror,ConnectionErrorretry warning,ConnectionErrorfinal errorclose()ruff format --check,ruff check, mypy baseline, and the public API snapshot check pass.pytest: 4716 passed. 3 AI-wrapper tests (langchain/test_callbacks.pyx2,anthropic::test_integration_stop_reason) fail onmaintoo, and are not related.📝 Checklist
If releasing new changes
sampo addto generate a changeset file🤖 Agent context
Autonomy: Fully autonomous
PosthogCeleryIntegration. That adds billable lifecycle events and edits task headers without consent. Rejected: only a warning log. It still leaves users with no events.Created with PostHog Desktop from this inbox report.
🤖 Generated with Claude Code
Origin
52cd20e