Repository navigation
Conversation
PostgreSQL Extension Dependency Analysis: PR #2424
SummaryNo extensions had dependencies with MAJOR version updates. Full Analysis ResultsPostgreSQL 15 Extension DependenciesPostgreSQL 17 Extension DependenciesOrioleDB 17 Extension Dependencies |
PostgreSQL Package Dependency Analysis: PR #2424
SummaryNo packages had MAJOR version updates. Full Analysis ResultsPostgreSQL 15 Dependency ChangesExtracting PostgreSQL 15 dependencies...
Runtime Closure Size
Raw Dependency ClosurePostgreSQL 17 Dependency ChangesExtracting PostgreSQL 17 dependencies...
Runtime Closure Size
Raw Dependency Closure |
This comment has been minimized.
This comment has been minimized.
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
from
September 8, 2026 01:13
48bf50f to
01c4e39
Compare
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
from
September 8, 2026 09:27
e610160 to
f5befb5
Compare
This comment has been minimized.
This comment has been minimized.
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
from
September 8, 2026 17:22
716aa2f to
b77c152
Compare
2 tasks done
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
4 times, most recently
from
September 8, 2026 22:51
31b661f to
99ac48b
Compare
brainrake
changed the title
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
2 times, most recently
from
September 9, 2026 00:52
0b4401c to
730068b
Compare
brainrake
changed the title
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
from
September 9, 2026 16:06
2944b3b to
eee3f28
Compare
brainrake
marked this pull request as ready for review
September 12, 2026 10:58
brainrake
marked this pull request as draft
September 18, 2026 01:50
brainrake
marked this pull request as ready for review
September 18, 2026 06:07
brainrake
marked this pull request as draft
September 20, 2026 13:36
brainrake
force-pushed
the
martonboros/mpg-12-site-env-update
branch
from
September 21, 2026 03:02
d974c98 to
774feca
Compare
brainrake
changed the title
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
mmlb
force-pushed
the
martonboros/mpg-12-site-env-update
branch
from
October 13, 2026 19:45
88c205a to
b4d3087
Compare
Collaborator
|
Rebased on top of latest develop so CI cleans up its AMIs |
Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
mmlb
reviewed
Oct 14, 2026
Comment on lines
+13
to
+18
| ```sh | ||
| nix-env --profile /nix/var/nix/profiles/site --set <store path> | ||
| /nix/var/nix/profiles/site/bin/activate | ||
| ``` | ||
|
|
||
| `--set` downloads the path if it is missing. `activate` sets up supautils, and gatekeeper where present, idempotently. |
Collaborator
There was a problem hiding this comment.
It would be good to call out what nix-env --set does and what activate does with a little more detail. Also what happens if nix-env --profile or activate fails?
| ## Roll back | ||
|
|
||
| ```sh | ||
| nix-env --profile /nix/var/nix/profiles/site --rollback |
Collaborator
There was a problem hiding this comment.
what about rolling back w/e activate does?
Collaborator
Author
There was a problem hiding this comment.
Indeed, forgot to cover that. Adding.
Comment on lines
+59
to
+63
| activate = writeHostBash "activate" '' | ||
| echo "Activating site profile." | ||
| echo "supautils at /nix/var/nix/profiles/site/lib/postgresql/supautils.so: picked up next session via session_preload_libraries, if dynamic_library_path includes /nix/var/nix/profiles/site/lib/postgresql." | ||
| echo "Site profile activated." | ||
| ''; |
Collaborator
There was a problem hiding this comment.
This doesn't do anything useful
Collaborator
Author
There was a problem hiding this comment.
Clarified in the PR description and docs: this is the entry point for idempotent updates / convergence. This is where we will move provisioning steps from salt and other places. Idempotency and rollback ensured through testing.
brainrake
commented
Oct 14, 2026
| # - Other Defaults - | ||
|
|
||
| #dynamic_library_path = '$libdir' | ||
| dynamic_library_path = '/nix/var/nix/profiles/site/lib/postgresql:$libdir' |
Collaborator
Author
There was a problem hiding this comment.
Load supautils from site profile. Safe migration story:
- Before any of this lands, supautils is in postgres libdir.
- AMI lands nix profile and config change. Loads supautils from site profile. Old AMIs don’t change.
- Salt writes it back to just libdir.
- Supautils is both in site profile and postgres libdir (written by salt to store). It is loaded from libdir.
- Flopping stops on new instances.
- Salt bootstrap lands. Old instances get nix profile bootstrapped but no config/package changes.
- Verify everyone has site profile.
- Salt config change lands. It loads supautils from site profile
- Verify config change landed cleanly.
- Clean up old salt and compatibility shims.
hitmands
reviewed
Oct 15, 2026
| with: | ||
| role-to-assume: ${{ secrets.DEV_AWS_ROLE }} | ||
| aws-region: ${{ env.AWS_REGION }} | ||
|
|
Contributor
There was a problem hiding this comment.
is this intentionally not using the same chain as above (L:162)?
- name: GitHub OIDC Auth
uses: aws-actions/configure-aws-credentials@7474bc4690e29a8392af63c5b98e7449536d5c3a # v4.3.1
with:
aws-region: us-east-1
role-to-assume: arn:aws:iam::279559813984:role/supabase-github-oidc-role
role-session-name: shared-services-jump
Collaborator
Author
There was a problem hiding this comment.
This is using a public bucket so the consumer doesn't need auth.
| role-to-assume: ${{ secrets.DEV_AWS_ROLE }} | ||
| aws-region: ${{ env.AWS_REGION }} | ||
|
|
||
| - name: Publish site-env store path |
Contributor
There was a problem hiding this comment.
ok, so this is why not using a json like the above catalog?
Comment on lines
+38
to
+44
| # postgres can load supautils via the site profile's dynamic_library_path | ||
| machine.succeed("install -d -o postgres -g postgres /tmp/pgdata") | ||
| machine.succeed("su postgres -c '${psql_17}/bin/initdb -D /tmp/pgdata'") | ||
| machine.succeed("install -o postgres -g postgres ${pgConf} /tmp/pgdata/postgresql.conf") | ||
| machine.succeed("su postgres -c '${psql_17}/bin/pg_ctl -D /tmp/pgdata -l /tmp/pg.log start'") | ||
| machine.succeed("su postgres -c '${psql_17}/bin/psql -h /tmp -d postgres -c \"select 1\"'") | ||
| machine.succeed("su postgres -c '${psql_17}/bin/pg_ctl -D /tmp/pgdata stop'") |
Contributor
There was a problem hiding this comment.
how do we make sure this is not coming from $libdir?
Collaborator
Author
There was a problem hiding this comment.
You are correct, the test doesn't show it. The intention was to remove supautils from $libdir but that needs to happen in a subsequent update after changes land in salt. I will add a test to prove it.
Comment on lines
+31
to
+32
| nix-env --profile /nix/var/nix/profiles/site --delete-generations +3 | ||
| nix-env --profile /nix/var/nix/profiles/default --delete-generations +2 |
Contributor
There was a problem hiding this comment.
this doesn't prune pg_ugprade, should it (I think so)?
Collaborator
Author
There was a problem hiding this comment.
I think not. Only a successful upgrade should remove that profile, and then GC will clean it.
Collaborator
Author
|
Thank you for the reviews! I've updated the PR description to answer some questions you might have. Please give it another look! |
…e site profile Refs: MPG-12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Moved |
mmlb
reviewed
Oct 16, 2026
| SITE_ENV_NAME="site-env-${POSTGRES_MAJOR_VERSION}" | ||
| SITE_ENV_PATH=$(nix eval --raw ".#${SITE_ENV_NAME}.outPath") | ||
|
|
||
| aws s3 cp - "s3://${{ secrets.SHARED_AWS_ARTIFACTS_BUCKET }}/nix-catalog/${GIT_SHA}-${SITE_ENV_NAME}-${SYSTEM}" <<< "$SITE_ENV_PATH" |
Collaborator
There was a problem hiding this comment.
We are going to want something similar for other updateable packages soon too right? This is almostnlike nix channels, for now can we drop the site-env from the name so its a directory and site-envs are files. So something like site-packages-${GIT_SHA}/
brainrake
commented
Oct 16, 2026
| # Site profile activation script. MUST STAY IDEMPOTENT! | ||
| activate = writeHostBash "activate" '' | ||
| echo "Activating site profile." | ||
| echo "supautils at /nix/var/nix/profiles/site/lib/postgresql/supautils.so: picked up next session via session_preload_libraries, if dynamic_library_path includes /nix/var/nix/profiles/site/lib/postgresql." |
Collaborator
Author
There was a problem hiding this comment.
Maybe warn if not used.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why?
Supautils is baked into postgres libdir, making it hard to update.
How
Publish supautils and gatekeeper in
site-envpackage sets and install intositeprofile.Update using
nix-envand idempotentactivatescript.What
site-env-*package sets including supautils/nix/var/nix/profiles/siteandactivateduring AMI bake.dynamic_library_pathinpostgresql.confnow includes supautils fromsitefirst, then$libdir.postgres-envas the fallback.ami-release-nix.ymlpublishes each store path string as plain text to the public artifacts bucket asnix-catalog/<sha>-site-env-<major>-<system>.latest-site-env-<major>-<system>.sha, feeding scheduled salt workflow.Why change
dynamic_library_path?The current path only includes
$libdir(default) which is the postgres install dir under/nixwhich is the reason for the store mutation hack in salt. Settingdynamic_library_pathis the intended way of loading extensions from outside the install dir. Supautils is such an extension as it is loaded per session and needs to be updated independently of postgres.Why nix profile and binary cache?
We already use nix profiles to provision software in the AMI and during upgrade. This PR fixes mutable supautils deploy while using nix profiles in the intended way, and unbocks immutable deploys.
What is the update process?
nix-env --set /nix/store/a1b2...-site-env-17 --profile /nix/var/nix/profiles/siteDownloads path if needed and sets the profile symlink to point to the given store path.
/nix/var/nix/profiles/site/activateActivates deployment, see below.
Rollback is similar, just deploy an earlier path.
Why an
activatescript?This is the main update mechanism, currently a no-op except printing status. It has strict requirements:
How is this safe?
For each PR, activation is ensured to be safe with the following test (forthcoming in #2563):
assert m0 == m3 and m1 == m2Safe rollback exceptions can be added by using a golden diff.
MPG-12