Skip to content

postgres: probe timeouts of 5 s (readiness) and 10 s (liveness), threshold 6 - #2496

Open
Davanum Srinivas (dims) wants to merge 2 commits into
agent-substrate:mainfrom
dims:postgres-probe-timeouts
Open

Davanum Srinivas (dims) wants to merge 2 commits into
agent-substrate:mainfrom
dims:postgres-probe-timeouts

Conversation

@dims

@dims Davanum Srinivas (dims) commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

The exec probes time out at the 1 s default under load; three misses restart postgres and ateapi loses its connections.

  • readiness: timeoutSeconds: 5, liveness: timeoutSeconds: 10 (a close wave under high load missed 4 of the 6 allowed liveness tries at 5 s), failureThreshold: 6 on both; periods unchanged.

@dims
Davanum Srinivas (dims) marked this pull request as ready for review October 12, 2026 15:15
@dims Davanum Srinivas (dims) changed the title postgres: give the probes a 5 s timeout and a threshold of 6 postgres: probe timeouts of 5 s (readiness) and 10 s (liveness), threshold 6 Oct 13, 2026
@dims
Davanum Srinivas (dims) force-pushed the postgres-probe-timeouts branch 4 times, most recently from b8da9f5 to ef809ca Compare October 18, 2026 21:04
@bowei Bowei Du (bowei) added area/dev-infra area/reliability kind/bug Something isn't working / bugfixes labels Oct 19, 2026
@dims
Davanum Srinivas (dims) force-pushed the postgres-probe-timeouts branch 10 times, most recently from b12976c to e17a776 Compare October 27, 2026 12:24
@dims
Davanum Srinivas (dims) force-pushed the postgres-probe-timeouts branch 3 times, most recently from cd5df1f to 24769a6 Compare October 31, 2026 17:05
Under load the exec probes timed out at their 1 s default; three misses
in a row restart postgres and take the control plane down mid-run.

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Under load the 5 s liveness probe missed 4 of its 6 allowed tries
during a close wave; one more miss would have restarted the database
mid-run. A slow database must leave the Service, not be killed, so
the readiness timeout stays at 5 s.

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dev-infra area/reliability kind/bug Something isn't working / bugfixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants