feat(projects): backfill and enforce membership in SQL by mzxchandra · Pull Request #8590 · simstudioai/sim

mzxchandra · 2026-10-02T22:32:41Z

Summary

Build on feat(projects): add project identity and lifecycle foundation #8580's merged additive schema and compatible application lifecycle handling. Run the data backfill and enforcement together as Drizzle SQL migration 0395; there is no standalone backfill CLI or writer activation flag.
Backfill only families missing membership, one bounded fork family per transaction. Leave complete existing Project assignments intact. Use per-environment advisory locks shared with compatible writers, nonblocking row/Project lock acquisition, and retries that release locks before waiting. Preserve committed progress and Project IDs across reruns; reject ambiguous legacy assignments.
Bound each family transaction to five seconds on PostgreSQL 17+. An overlong family aborts the migration connection and rolls back that family; previously committed families remain safe to resume.
Validate without write-blocking table locks. Install lifecycle triggers in a separate brief transaction with nonblocking table-lock acquisition and a five-second transaction-wide timeout, then validate again before journaling success. PostgreSQL lock inspection verifies that a stalled migration client releases all three table locks; a busy-table check verifies prompt failure and cleanup.
Independently check the compatible ECS rollout before migrating an existing hosted database. Relevant old worker runs still require operator confirmation. Project API exposure remains separately controlled.
Keep workflow creation and restore guarded by workspace share locks in feat(projects): add project identity and lifecycle foundation #8580. Do not install workflow triggers or serialize workflow writes on the Project row. Project/environment membership and archive constraints remain enforced.
Seed and remove integration-test Projects atomically with their environments; exercise schema-push and full-migration provisioning in CI.

Merge only after #8580 is deployed and pre-foundation servers and relevant workers have drained. The PR2 migration then performs backfill and enforcement before PR2's app version is served. Verify the workflow role's AWS read permissions before rollout; live AWS verification has not been performed.

Review scope: 97 of the 105 changed files are integration/E2E tests or fixture helpers. They now seed workspace/Project membership together and clean up empty fixture Projects under the enforced schema. No production knowledge-base behavior changes are included. The other eight files contain SQL/Drizzle metadata, the schema annotation, and deployment preflight wiring.

Type of Change

New feature

Testing

30 SQL migration checks passed against disposable PostgreSQL, covering real Drizzle journaling, replay, conflict remediation, concurrent writes/forks, cancellation, and an intentionally stalled family that exceeds the transaction timeout. A multi-family fixture covers separately committed batches.
All 181 database-package integration checks passed against the full migration sequence, including the 30 Project SQL checks. Complete existing Projects are covered separately from legacy families, including Projects above the legacy batch limit.
All 82 targeted application integration checks passed through both full migrations and schema-push provisioning: 31 Project foundation checks, 21 fork-sync checks, 28 execution/archive provenance checks, and two desktop executor checks. Foundation checks use a private migrated database; temporary legacy trigger removal cannot affect shared suites.
A bulk archive performs one full Project validation instead of 806 in the pre-fix regression. Immediate constraints and repeatable-read concurrent archival remain covered. Four preflight CLI regressions reject stale image acknowledgments and newer deployments in unordered histories.
Full repository tests, all 26 workspace type-checks, lint, all 58 repository audits, migration safety, generated-artifact checks, and Drizzle schema-drift validation passed. Regression cases were verified failing before their fixes.
Hosted CI passed at ef5498a25f: build, lint/tests, PostgreSQL integration through both migration and schema-push provisioning, and real HTTP E2E. The three newly merged fixtures now create and clean up Project membership. Locally, both affected PostgreSQL suites passed all 7 checks and the HTTP/CLI stop-after suite passed all 11 checks, including cleanup; full repository tests, type-checks, lint and all 58 audits also passed. Greptile reports 5/5 and both reviewers have zero unresolved threads.
Live AWS rollout verification and hosted migration execution have not been performed.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Added/updated Project tests pass the test-audit authoring gate and targeted runs
Final hosted test/build workflow passes, including both database provisioning paths and HTTP end-to-end tests
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-10-02T22:32:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Oct 6, 2026 3:53am UTC

…ty-enforcement # Conflicts: # apps/sim/lib/projects/__integration__/foundation.integration.ts

…ty-enforcement

mzxchandra · 2026-10-05T05:18:51Z

@greptile

mzxchandra · 2026-10-05T05:18:53Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-05T05:18:59Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 105 files

_{Fix all with cubic | Re-trigger cubic}

mzxchandra · 2026-10-05T05:33:36Z

@greptile

mzxchandra · 2026-10-05T05:33:38Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-05T05:33:43Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 105 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic}

cubic-dev-ai

No issues found across 105 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

mzxchandra · 2026-10-06T01:01:22Z

@greptile

mzxchandra · 2026-10-06T01:01:23Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T01:01:28Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 105 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic}

mzxchandra · 2026-10-06T01:24:42Z

@greptile

mzxchandra · 2026-10-06T01:24:43Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T01:24:49Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 108 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic}

mzxchandra added 7 commits October 2, 2026 00:31

feat(projects): add project identity and lifecycle foundation

00a83d0

feat(projects): create projects with their initial environment

aa9b415

docs(projects): record project files follow-up

91cba23

docs(projects): explain project and workspace creation flows

0980ad1

fix(projects): stage activation after compatible writers deploy

a14b448

feat(projects): enforce membership after the staged backfill

5121347

fix(projects): retry writes after concurrent membership changes

523338a

mzxchandra mentioned this pull request Oct 2, 2026

feat(projects): add project identity and lifecycle foundation #8580

Merged

6 tasks done

vercel Bot deployed to Preview October 2, 2026 22:34 View deployment

fix(projects): align integration fixtures with membership constraints

dc5e301

vercel Bot deployed to Preview October 2, 2026 23:01 View deployment

mzxchandra added 5 commits October 2, 2026 16:34

refactor(projects): prepare compatible writers for the SQL backfill

c1c8e5d

Merge branch 'feat/project-entity-foundation' into codex/project-enti…

2e94a13

…ty-enforcement # Conflicts: # apps/sim/lib/projects/__integration__/foundation.integration.ts

refactor(projects): backfill through bounded SQL migration batches

5e11d7d

fix(projects): clean up automatically created fixture Projects

99189f2

Merge branch 'feat/project-entity-foundation' into codex/project-enti…

78f773d

…ty-enforcement

vercel Bot deployed to Preview October 2, 2026 23:54 View deployment

fix(projects): backfill only families missing membership

2ecbfdf

vercel Bot deployed to Preview October 3, 2026 00:00 View deployment

mzxchandra changed the title ~~feat(projects): enforce membership after the staged backfill~~ feat(projects): backfill and enforce membership in SQL Oct 3, 2026

mzxchandra added 2 commits October 2, 2026 17:19

chore(projects): merge staging into foundation

30bc9d8

chore(projects): merge updated foundation into enforcement

b5cc3c4

vercel Bot deployed to Preview October 3, 2026 00:24 View deployment

fix(projects): bound the trigger installation lock window

2afc652

vercel Bot deployed to Preview October 3, 2026 01:06 View deployment

mzxchandra added 4 commits October 2, 2026 18:21

fix(workflows): guard restore against concurrent workspace archive

6dfbc4f

fix(projects): keep workflow lifecycle guards at workspace scope

8965210

fix(workflows): guard restore against concurrent workspace archive

552ce68

Merge branch 'feat/project-entity-foundation' into codex/project-enti…

8be18e5

…ty-enforcement

vercel Bot deployed to Preview October 5, 2026 05:20 View deployment

cubic-dev-ai Bot reviewed Oct 5, 2026

View reviewed changes

Comment thread packages/db/scripts/project-contract.integration.ts Outdated

test(projects): verify concurrent lifecycle lock contention

8f36f2d

vercel Bot deployed to Preview October 5, 2026 05:35 View deployment

cubic-dev-ai Bot reviewed Oct 5, 2026

View reviewed changes

mzxchandra marked this pull request as ready for review October 5, 2026 05:38

cubic-dev-ai Bot reviewed Oct 5, 2026

View reviewed changes

Base automatically changed from feat/project-entity-foundation to staging October 5, 2026 17:40

mzxchandra added this pull request to stack #8630 October 5, 2026 18:24

chore(projects): merge staging into membership enforcement

7ec83f0

vercel Bot deployed to Preview October 6, 2026 01:04 View deployment

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread .github/scripts/check-project-rollout.py

test(projects): update new workspace fixtures for enforcement

ef5498a

vercel Bot temporarily deployed to Preview October 6, 2026 01:24 Inactive

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

mzxchandra added 2 commits October 5, 2026 20:32

chore(projects): sync enforcement migrations with staging

41b3a28

chore(projects): follow staging table migration

10a5f06

vercel Bot deployed to Preview October 6, 2026 03:53 View deployment

Conversation

mzxchandra commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mzxchandra commented Oct 5, 2026

Uh oh!

mzxchandra commented Oct 5, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 5, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mzxchandra commented Oct 5, 2026

Uh oh!

mzxchandra commented Oct 5, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 5, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mzxchandra commented Oct 6, 2026

Uh oh!

mzxchandra commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mzxchandra commented Oct 6, 2026

Uh oh!

mzxchandra commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mzxchandra commented Oct 2, 2026 •

edited

Loading

vercel Bot commented Oct 2, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading