Repository navigation
8 - FEAT: Contextual Help - #3646
Richard Lundeen (richlundeen) wants to merge 88 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Move history into feature routes, add stored ID lookup, and retain the current chat draft. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve discard blocking before resetting the composer and use shared encoded continuation and scenario routes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep an explicit not-found page and Home recovery link at unknown URLs. Preserve the last valid chat and dataset views without changing legacy aliases or Registry index behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Prefer the existing retained draft when entering Chat from another view. Keep an already open chat in place so live work is not replaced by an older snapshot. Cover the workflow with the real History table, Chat tabs, composer and reset controls. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Move score matching into Scorer.find_matching_scores so reuse compares the expectation score_async prepares, share chat evaluation and bounded publication between Send and Apply, project the latest-response result, outcome, and canonical response ID from the backend, and keep Apply status in ChatWindow. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Show compact scan results, saved seed and component identities, live read-only scan conversations, and faithful saved launch configuration. Reuse SDK progress, scoring, and runtime ownership. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Expose nested active attempts and chronological current conversations. Restore scoped success rates and compact operational notices, isolate timing updates, validate saved rerun references, restore drill-down focus, and remove obsolete paired frontend surfaces while retaining public analytics contracts. Cancellation persistence and outcome classification are excluded pending user decision. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
HELD PROPOSAL, not approved for publication. Reuses ERROR/CancelledError records to retain interrupted conversations and joins publication before releasing execution. This changes historical attempt/error counts and all-outcome denominators; user review must choose this policy or attribution-only persistence before release. Keep this commit separate from confirmed repairs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the unapproved ERROR cancellation proposal from the active candidate. The separate proposal commit and cancellation-semantics.json remain available for joint review. Attribution-only preparation must preserve outcome, completion, denominator and retry semantics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Prepare attribution-only option A: SDK workers collect observed conversation links and finalize existing ScenarioResult metadata after draining. Scoped reads verify Conversation.attack_result_id. Restore read-only stopped-chat reload without creating AttackResults or changing completion, outcome, denominator or retry semantics. Publication remains held for joint review and the corrected predecessor. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Expose saved provenance links without inventing result rows or rates, and describe a cancelled scan truthfully rather than reporting zero attack errors as a failure. Attribution-only candidate remains local and unpublished. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1412c1f to
4a3fb44
Compare
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
4a3fb44 to
8ab525d
Compare
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use real summaries and saved operation links. Keep intentional New attack resets separate from general navigation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the messages GET aligned with current attack score, response, outcome, and failure state across successful and failed sends. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve the original PR head and fresh main ancestry while incorporating dependency preparation and interaction contracts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Select scenario history by activity in the database while preserving default history cursor semantics and the mobile registry audit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Disambiguate outcome regression selectors and supply request-converter help in the registry fixture. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Addressed both comments in 9a0d77b (repair commit dc19b47). Native help-note and tooltip content now use the existing editor-controls marker, with dirty-draft pointer/selection regressions and genuine outside-save behavior preserved. The estimate and run preview now describe a configured dataset selection limit and explain that actual selection may be smaller. The count calculation is unchanged. |
Preserve FIFO handoff coverage and detect queue clicks that reload the document. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; it does not authorize merge.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Composite scorers can bypass a child ConversationScorer’s snapshot validation, allowing an outdated verdict to be reused and displayed as current after a user turn is appended.
1 open issue
What changed in this PR
Adds contextual help to CoPyRIT’s chat, scanner, and Registry interfaces while preserving visible guidance and normal field interactions. This cumulative diff also includes workflow navigation, scoring, and scan-history changes from preceding layers.
Changes:
- Introduces shared help definitions, information controls, accessible field descriptions, and touch-friendly layouts.
- Organizes navigation around Attacks and Scanner, updates links, and preserves draft and preference behavior.
- Extends chat scoring, component identities, and scan progress metadata, with corresponding tests and documentation.
| File | Description |
|---|---|
| tests/unit/registry/test_scenario_registry.py | Tests scorer and technique identity metadata. |
| tests/unit/memory/test_scenario_history.py | Tests history filter forwarding. |
| tests/unit/executor/attack/core/test_attack_executor.py | Tests active execution snapshots and cancellation cleanup. |
| tests/unit/backend/test_technique_service.py | Tests canonical technique identities. |
| tests/unit/backend/test_scenario_service.py | Tests catalog identity behavior. |
| tests/unit/backend/test_scenario_run_routes.py | Tests history filters and ID validation. |
| tests/unit/backend/test_scenario_progress_read_model.py | Tests seed provenance and nested scorer identities. |
| tests/unit/backend/test_message_send_service.py | Updates conversation retrieval mocks. |
| tests/unit/backend/test_finding_export.py | Updates exported conversation link expectations. |
| tests/unit/backend/test_attack_service.py | Tests shared score forwarding. |
| tests/unit/backend/test_api_routes.py | Tests Apply scorer response mapping. |
| pyrit/score/scorer.py | Adds stored-score matching APIs. |
| pyrit/score/conversation_scorer.py | Validates conversation snapshots when matching scores. |
| pyrit/scenario/core/scenario_technique.py | Exposes factory-backed technique identity. |
| pyrit/registry/components/scenario_registry.py | Includes identities in scenario metadata. |
| pyrit/models/score/scorable.py | Adds evidence identity comparison. |
| pyrit/models/scenario_progress.py | Extends progress and conversation metadata. |
| pyrit/models/catalog/scenario.py | Adds catalog identity fields. |
| pyrit/models/__init__.py | Exports new progress models and constants. |
| pyrit/memory/memory_interface.py | Forwards history reference and sorting options. |
| pyrit/executor/attack/core/attack_strategy.py | Tracks objective conversation activity. |
| pyrit/executor/attack/compound/sequential_attack.py | Exposes active child executions. |
| pyrit/backend/services/scenario_progress_read_model.py | Projects seed identifiers into progress. |
| pyrit/backend/services/manual_send_scheduler.py | Retains write ownership through cancellation. |
| pyrit/backend/services/finding_export.py | Uses updated chat conversation routes. |
| pyrit/backend/services/attack_service.py | Returns conversation scoring state. |
| pyrit/backend/routes/attacks.py | Adds the Apply scorer endpoint. |
| pyrit/backend/README.md | Documents exposed technique identity JSON. |
| pyrit/backend/models/techniques.py | Adds canonical technique identity. |
| pyrit/backend/models/scores.py | Defines Apply scorer request and response models. |
| pyrit/backend/models/attacks.py | Adds conversation score and outcome fields. |
| pyrit/backend/mappers/technique_mappers.py | Maps factory identity into responses. |
| pyrit/backend/mappers/attack_mappers.py | Shares batched message-score retrieval. |
| pyrit/backend/mappers/__init__.py | Exports the score retrieval helper. |
| frontend/vite.config.ts | Updates dependency transform configuration. |
| frontend/src/utils/targetIdentity.ts | Preserves source references for bound targets. |
| frontend/src/utils/targetIdentity.test.ts | Tests source and execution identity distinctions. |
| frontend/src/utils/scorerPresentation.ts | Validates saved chat scorer references. |
| frontend/src/utils/scorerPresentation.test.ts | Tests scorer preference eligibility. |
| frontend/src/utils/scenarioRunProgress.ts | Tracks active and recorded conversations. |
| frontend/src/utils/routeParams.ts | Updates workflow routes and adds run helpers. |
| frontend/src/utils/chatDraft.ts | Recognizes additional retained draft settings. |
| frontend/src/utils/attackConversationRoute.cases.json | Updates shared conversation route fixtures. |
| frontend/src/styles/touchTargets.ts | Exposes a reusable narrow-screen query. |
| frontend/src/services/api.ts | Extends scoring and scan APIs. |
| frontend/src/services/api.test.ts | Tests activity-sort forwarding. |
| frontend/src/hooks/useTour.ts | Updates route examples in a comment. |
| frontend/src/hooks/useTargetRegistry.ts | Tracks generation-aware empty state. |
| frontend/src/hooks/useTargetPreferences.ts | Resolves last-used and adversarial targets. |
| frontend/src/hooks/useScenarioLaunchSeed.ts | Loads saved run configuration for reruns. |
| frontend/src/hooks/useScenarioLaunchSeed.test.ts | Tests rerun loading and local attribution edits. |
| frontend/src/hooks/useRunTechniqueEditor.ts | Restores initial technique overrides. |
| frontend/src/hooks/useMediaQuery.ts | Adds reactive media-query subscription. |
| frontend/src/hooks/useMediaQuery.test.ts | Tests subscription updates and cleanup. |
| frontend/src/hooks/useConversationDraft.ts | Uses execution identity for dirty tracking. |
| frontend/src/hooks/useConversationDraft.test.ts | Tests bound-target dirty tracking. |
| frontend/src/hooks/useChatDraftHandoff.tsx | Rejects superseded draft snapshots. |
| frontend/src/components/Tour/tourSteps.ts | Updates workflow and preference guidance. |
| frontend/src/components/Sidebar/Navigation.styles.ts | Styles expanded navigation and active states. |
| frontend/src/components/ScenarioSuites/suiteRoutes.ts | Moves suite progress under scanner history. |
| frontend/src/components/ScenarioSuites/suiteRoutes.test.ts | Tests updated suite routes. |
| frontend/src/components/ScenarioSuites/ScenarioSuiteExecutionPage.tsx | Updates navigation links and heading levels. |
| frontend/src/components/ScenarioSuites/ScenarioSuiteExecutionPage.test.tsx | Updates route and link assertions. |
| frontend/src/components/Scenarios/ScenarioRunTiming.tsx | Displays elapsed time and estimates. |
| frontend/src/components/Scenarios/ScenarioQueuePage.tsx | Adds a dedicated scan queue page. |
| frontend/src/components/Scenarios/ScenarioQueue.tsx | Uses client-side run links. |
| frontend/src/components/Scenarios/ScenarioDetail.styles.ts | Supports responsive identity disclosures. |
| frontend/src/components/Scenarios/ScenarioCatalog.tsx | Updates launch routes and catalog wording. |
| frontend/src/components/Scenarios/ScenarioCatalog.test.tsx | Tests revised catalog labels and links. |
| frontend/src/components/Scenarios/PairedOutcomes.styles.ts | Removes paired-outcome styles. |
| frontend/src/components/ScenarioPresets/ScannerVisibilityLabel.tsx | Uses shared contextual help. |
| frontend/src/components/ScenarioPresets/presetRoutes.ts | Updates preset launch routes. |
| frontend/src/components/ScenarioPresets/presetRoutes.test.ts | Tests updated preset routes. |
| frontend/src/components/Registry/TechniqueRegistry.styles.ts | Removes obsolete detail styles. |
| frontend/src/components/Registry/ScorerRegistry.test.tsx | Tests identity disclosures and last-used preferences. |
| frontend/src/components/Registry/ScorerRegistry.styles.ts | Removes superseded identity styles. |
| frontend/src/components/Registry/ScenarioPresetRegistry.tsx | Updates the scanner launch link. |
| frontend/src/components/Registry/ScenarioPresetRegistry.test.tsx | Updates scanner link assertions. |
| frontend/src/components/Registry/RegistryLayout.tsx | Adds explanatory tab tooltips. |
| frontend/src/components/Registry/RegistryIdentityRow.tsx | Adds expandable identity JSON rows. |
| frontend/src/components/Registry/RegistryIdentityRow.test.tsx | Tests keyboard disclosure and independent rows. |
| frontend/src/components/Registry/RegistryIdentityRow.styles.ts | Styles shared identity disclosures. |
| frontend/src/components/Registry/Registry.styles.ts | Removes obsolete converter styles. |
| frontend/src/components/Registry/ConverterRegistry.styles.ts | Adds converter page and control styles. |
| frontend/src/components/Parameters/ReferenceField.tsx | Supports shared field help. |
| frontend/src/components/Parameters/ParameterField.test.tsx | Tests help and visible format guidance. |
| frontend/src/components/Operations/OperationDetailPage.tsx | Links separate attack and scanner histories. |
| frontend/src/components/Operations/OperationDetailPage.test.tsx | Tests operation-filtered history links. |
| frontend/src/components/Operations/FindingEvidenceList.test.tsx | Updates evidence link expectations. |
| frontend/src/components/Layout/PageNotFound.tsx | Adds an unknown-route page. |
| frontend/src/components/Layout/PageNotFound.styles.ts | Styles the unknown-route page. |
| frontend/src/components/Layout/MainLayout.test.tsx | Tests sidebar persistence without workspace remounting. |
| frontend/src/components/Layout/MainLayout.styles.ts | Adds responsive sidebar and viewport layout. |
| frontend/src/components/Layout/FeatureNavigation.tsx | Adds workflow tabs and New attack action. |
| frontend/src/components/Layout/FeatureNavigation.styles.ts | Styles workflow navigation. |
| frontend/src/components/History/scenarioHistoryFilters.ts | Adds URL-backed reference ID filtering. |
| frontend/src/components/History/scenarioHistoryFilters.test.ts | Tests reference ID round-tripping. |
| frontend/src/components/History/ScenarioHistory.styles.ts | Styles ID lookup and history layout. |
| frontend/src/components/History/HistoryPage.tsx | Removes the shared history wrapper. |
| frontend/src/components/History/HistoryPage.test.tsx | Removes shared history wrapper tests. |
| frontend/src/components/History/HistoryPage.styles.ts | Removes shared history wrapper styles. |
| frontend/src/components/History/AttackHistory.tsx | Directs the empty-state action to Chat. |
| frontend/src/components/History/AttackHistory.test.tsx | Tests revised empty-state navigation. |
| frontend/src/components/HelpField.tsx | Keeps help available as field descriptions. |
| frontend/src/components/helpDefinitions.ts | Centralizes common term definitions. |
| frontend/src/components/Converters/ConverterChainField.tsx | Adds explicit converter-chain help. |
| frontend/src/components/Converters/ConverterChainField.test.tsx | Tests converter help surfaces. |
| frontend/src/components/ContextualHelpLabel.tsx | Combines term hover and information controls. |
| frontend/src/components/ContextualHelpLabel.styles.ts | Styles help content and touch controls. |
| frontend/src/components/Configuration/Configuration.test.tsx | Updates scanner navigation in tests. |
| frontend/src/components/Configuration/Configuration.reload-guard.test.tsx | Updates reload-guard navigation fixtures. |
| frontend/src/components/Config/TargetSelect.tsx | Supports accessible target field help. |
| frontend/src/components/Config/TargetConfig.tsx | Adds Registry help and removes objective defaults. |
| frontend/src/components/Chat/SystemPromptSetup.tsx | Adds system-prompt explanation. |
| frontend/src/components/Chat/ObjectiveHeader.styles.ts | Enlarges result control touch targets. |
| frontend/src/components/Chat/MultiSendSettings.tsx | Adds repetition and converter help. |
| frontend/src/components/Chat/ChatWindow.styles.ts | Adjusts narrow-screen chat layout. |
| frontend/src/components/Chat/ChatTargetPicker.tsx | Associates Objective target help with the selector. |
| frontend/src/components/Chat/ChatTargetPicker.test.tsx | Tests accessible naming and label focus. |
| frontend/src/components/Chat/ChatTargetPicker.styles.ts | Allows target label and selector wrapping. |
| frontend/src/components/AttackResults/attackAttemptFormatting.ts | Updates success and rationale presentation. |
| frontend/src/components/AttackResults/AttackAttemptDetails.styles.ts | Emphasizes conversation links. |
| frontend/src/auth/AuthProvider.tsx | Updates a route example in a comment. |
| frontend/src/auth/AuthProvider.test.tsx | Tests redirects using updated chat routes. |
| frontend/README.md | Documents contextual help interactions. |
| frontend/playwright.paired-outcomes.config.ts | Removes dedicated paired-outcome configuration. |
| frontend/playwright.config.ts | Updates test selection and preference fixtures. |
| frontend/package.json | Removes the paired-outcome test command. |
| frontend/e2e/themes.spec.ts | Updates themed route coverage. |
| frontend/e2e/sidebar.spec.ts | Tests sidebar persistence and narrow navigation. |
| frontend/e2e/routing.spec.ts | Updates workflow routing checks. |
| frontend/e2e/labels-operation-picker.spec.ts | Updates operation and history response fixtures. |
| frontend/e2e/history.spec.ts | Tests workflow history and empty-state navigation. |
| frontend/e2e/flows.spec.ts | Uses direct chat target selection. |
| frontend/e2e/errors.spec.ts | Updates target setup and navigation checks. |
| frontend/e2e/converters.spec.ts | Updates target selection and history navigation. |
| frontend/e2e/compatibility.spec.ts | Uses updated navigation selectors. |
| frontend/e2e/chat-scoring.spec.ts | Updates scoring responses and chat routes. |
| frontend/e2e/chat-recovery.spec.ts | Updates target setup and recovery navigation. |
| frontend/e2e/api.spec.ts | Updates navigation readiness assertions. |
| frontend/e2e/analytics.spec.ts | Updates saved-result route expectations. |
| frontend/e2e/_fixtures.ts | Adds shared chat target selection. |
| doc/code/framework.md | Documents score matching and chat scoring ownership. |
| .github/workflows/frontend_tests.yml | Removes the paired-outcome browser-test job. |
Review effort: Balanced
| Returns: | ||
| Matching scores in newest-first order, including undetermined results. | ||
| """ | ||
| return self.find_matching_scores(scores=scores, scorable=scorable, expectation=expectation) |
There was a problem hiding this comment.
Fixed in 71614ae. Wrapper scores now retain child participation by tree node, so composite freshness checks only children that contributed a result, and nested fallbacks follow their own participating primary/fallback branches instead of conflicting root or prefixed metadata. Added mixed fallback-sibling, nested-fallback, non-applicable-child, missing-observation, and backend Apply/display regressions. The scoped scorer/backend and matching/wrapper suites pass. SQLite-backed regressions also verify persisted snapshot-free fallback collisions reuse and project the same root score without rescoring after an upgrade, while missing required legacy conversation provenance safely triggers reevaluation.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Validate current conversation snapshots through same-input scorer wrappers and keep tour navigation on settled tooltips. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Replying to #3646 (review) Addressed in 71614ae. This umbrella is the same wrapped ConversationScorer freshness cause as inline:4439940924. Per-node participation now preserves actual fallback and composite branches, including empty/non-applicable children, while stale or missing required conversation evidence remains a safe cache miss. The scoped scorer/backend and matching/wrapper regressions pass. SQLite-backed regressions also verify persisted snapshot-free fallback collisions reuse and project the same root score without rescoring after an upgrade, while missing required legacy conversation provenance safely triggers reevaluation. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Adds contextual help through native Fluent information controls, while retaining term-text hover and keyboard/touch access. Keeps essential guidance visible and corrects target labels and empty-state navigation.
Changes:
str; show unavailable-estimate reasons without requiring a tooltip.Layer 8/10 | Previous: #3645 | Final: #3648 | Blocked by: #3645
Diff is cumulative until preceding layers merge.
Before (original GUI)
The original scanner form keeps Objective Target and objective explanations inline beneath their fields.
After
Objective target help uses a native information button beside the existing selector.
Simulation terms keep help beside the relevant control.
Adversarial-target help explains its role next to the configuration field.
The narrow objective editor and composer remain accessible without overlapping controls.
Video
contextual-help.webm