Skip to content

Preserve checklist state when saving a starting step - #999

Merged
thomasluizon merged 1 commit into
mainfrom
fix/ticket-1857-g9a-small-step
Oct 29, 2026
Merged

thomasluizon merged 1 commit into
mainfrom
fix/ticket-1857-g9a-small-step

Conversation

@thomasluizon

Copy link
Copy Markdown
Owner

Tracks thomasluizon/orbit-tickets#1857.

Saving a starting step could replace the full checklist and uncheck an existing item. update_checklist now accepts item_to_add, previews the stored list plus the unchecked step, and dispatches the existing append command after approval. Full-list additions must keep every stored item's text, order and checked state. The operation gateway checks these arguments before holding a preview.

The change lives in UpdateChecklistTool.cs, PendingOperationChangePreviewer.Checklist.cs and the checklist argument-check entry in AgentOperationExecutor.cs. This keeps the model responsible for tool selection while the server owns preservation. Existing validators, domain guards, ownership checks and approval binding remain on the path.

The English checklist-save setup says "I can't get myself to drink water." Its captured attempt records no tool calls and a missing successful suggest_breakdown. The tool description now explicitly covers inability to do an activity and requests for a first step, even without the word start. The shared prompt changes only the checklist-save instruction. No message pattern selects or rewrites a tool or substitutes an answer.

AnswerComposition.md and the tool README describe the append contract. The capability snapshot records its optional input; the gap report is unchanged. Release generation left all four committed OpenAPI artifacts unchanged. Paired work remains in thomasluizon/orbit-tickets#1866 and thomasluizon/orbit-tickets#1867.

Assumptions

  • Add optional item_to_add to the existing tool rather than introducing another tool or endpoint; a growing full replacement cannot also edit stored items.
  • Show the derived merged checklist as read-only in append previews rather than allowing full-list edits to change the append intent; ordinary replacement previews keep their existing editing support.

Test evidence

Before changing tests or implementation, dotnet test tests/Orbit.Application.Tests --filter FullyQualifiedName~OneSmallStep_ChecklistSavePreservesExistingItemsAndWaitsForApproval --verbosity quiet exited 0: both existing cases passed with the defect present because their model arguments already preserved the list.

Before the fix, dotnet test tests/Orbit.Application.Tests --no-build --filter FullyQualifiedName~OneSmallStep_CapturedChecklistAdditionCannotChangeStoredItems --logger 'console;verbosity=normal' exited 1: both composition modes failed because a held preview contained the captured unchecked replacement. The replay uses the real pt-BR r3 arguments, replacing only the redacted habit ID with the seeded ID.

After the fix, the regression verifies a typed denial of the captured checked-state change, zero pending operations, an append-only retry, approval, and independent habit-detail readback. The captured duplicate text is appended once unchecked while the existing checked occurrence remains checked. Separate tool tests reject changed text, reordered entries, malformed inputs, foreign targets and full checklists. English and pt-BR save tests cover both the new append input and valid full-list additions.

Checks completed by exit code:

Check Result
dotnet build Orbit.slnx -c Release --verbosity quiet exit 0, zero errors
Focused starting-step and checklist tool tests exit 0, 93 passed
dotnet test tests/Orbit.Domain.Tests -c Release --no-build --verbosity quiet exit 0, 1215 passed
dotnet test tests/Orbit.Application.Tests -c Release --no-build --verbosity quiet exit 0, 9190 passed
dotnet test tests/Orbit.Infrastructure.Tests -c Release --no-build --verbosity quiet exit 0, 6600 passed and 14 skipped
dotnet test tests/Orbit.Analyzers.Tests -c Release --no-build --verbosity quiet exit 0, 40 passed
python3 .github/scripts/openapi_artifacts.py freshness exit 0, every supported artifact matches generation
python3 .github/scripts/openapi_artifacts.py plan --base origin/main exit 0, supported comparison plan generated
node tools/capability-inventory.mjs --check exit 0, no drift
Commit hooks and git diff --check exit 0

Copy review

Every added or changed model-facing string has its English runtime text and pt-BR review translation in src/Orbit.Api/OpenApi/AnswerComposition.md, under "One small starting step": the starting-activity prompt, suggest_breakdown description, update_checklist description, both checklist input descriptions, conflicting-input and type errors, and the addition-preservation guard. Existing visible step labels and action messages are unchanged.

Evidence boundary

The committed fixture tests/Orbit.Application.Tests/Fixtures/ChecklistSaves/composition-one-small-step-save-checklist-pt-BR-r3.json is extracted from the coordinator's captured g9a attempt, retaining its case, repeat, failure reason, fixture facts and tool calls. The English setup request was read from orbit-ui-mobile at origin/main:evals/astra/composition-scenarios.yaml; its r1 attempt contains an empty toolCalls array, so no successful call was invented for replay. This change adds no reads of external service or package response fields. Live pass^3 evals remain the coordinator's verification.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes across the checklist tool, operation gateway, approval previews, starting-step instructions, regression tests, and shared consumer preview handling.

  • Append-only saves: item_to_add uses the existing append command, preserving stored text, order, and checked state while adding the new item unchecked.
  • Replacement safeguards: Growing full-list replacements must preserve the stored prefix, with argument validation running before a preview is held.
  • Approval previews: Append previews show the merged checklist as read-only while retaining append intent for execution; existing revision and stale-preview checks remain in place.
  • Regression coverage: Captured pt-BR failure replay and bilingual save tests cover preservation and approval. Both local test selections passed: 88 starting-step/tool tests and 617 preview, revision, and chat approval tests. Live model pass^3 evaluation was not run in this review.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@sonarqubecloud

Copy link
Copy Markdown

@thomasluizon
thomasluizon merged commit 3277ebf into main Oct 29, 2026
27 checks passed
@thomasluizon
thomasluizon deleted the fix/ticket-1857-g9a-small-step branch October 29, 2026 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant