Skip to content

CI: verify onboard feature execution within environment jobs - #2924

Open
zxx0288 wants to merge 1 commit into
hw-native-sys:mainfrom
zxx0288:support/verify-onboard-feature-coverage
Open

zxx0288 wants to merge 1 commit into
hw-native-sys:mainfrom
zxx0288:support/verify-onboard-feature-coverage

Conversation

@zxx0288

@zxx0288 zxx0288 commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Review follow-up (473da4e)

  • Permit the exact A2/A3 counting-window skip in the Daily main sweep without enabling PMU globally. The dedicated DFX smoke still owns actual PMU acceptance.
  • Both admin comment channels pin the upstream PR merge SHA after verifying its base/head parents. Missing, conflicting or stale snapshots reject admission; the gate summary records head/base/merge and the A5 report identifies the tested merge. Older heads therefore receive current-main CI interfaces after migration, without silently falling back to weaker verification. Legacy delegates still support the opposite pre-merge caller direction.
  • Repair the example README's job names, section link and result path. Inventory policy errors persist as policy_error JSON and exit nonzero rather than INTERNALERROR. Document and test that auxiliary probes do not qualify Kernel capture gaps.
  • 259 related CPU regressions passed; 14 new negative controls failed on the prior implementation. Ruff, Pyright, Markdown and actionlint passed (explicit existing self-hosted labels; shellcheck/pyflakes integrations disabled).
  • Rebased onto main d24e999, retaining the separately merged handshake repair and A5 async work. Add Program/eager metadata for 12 newly introduced standalone A5 tests; their test bodies are unchanged.

Earlier CI evidence below is historical and does not qualify this updated head. New CI and administrator-controlled A5/Daily validation remain required; no comment-triggered hardware run was posted by this repair. The updated comment definitions only become active on the default branch after merge; candidate-ref Daily workflow_dispatch is needed to validate candidate workflows before then. PR remains draft.

An onboard job's exit status alone does not show which promised features actually ran. This change names the framework suite by its dependency environment and verifies declared feature groups against source-bound execution receipts inside the existing default and torch_npu jobs.

Related to #2904.

Changes

  • Rename the framework suite to st-framework-<arch>, its reusable workflow to _st-framework.yml, and setup action to setup-framework. Update PR, Daily, emergency and A5 comment callers, artifact paths, contracts and documentation together. Preserve trigger gates, device leases, manual selection and A2/A3 main/SDMA/fault isolation.
  • Preserve thin delegating setup/identity entry points for comment workflows loaded from the default branch before migration. They forward the platform and CANN output and preserve the old caller's identity-output path without duplicating setup logic or execution suites. Document their removal condition and the fact that an older caller only qualifies its own verification contract.
  • Declare Program/Kernel and eager/capture-replay explicitly. Argument dumps remain eager; the common stream probe does not qualify HBG end-to-end capture. Include native-submission and Host-domain consumers introduced on main, preserving the existing A5 Host-domain quarantine.
  • Require applicable feature groups and their mandatory collected members. Missing classifications, ad-hoc deselection, lost/duplicate/foreign results and unexpected skips fail verification. Default skip exceptions require explicit platforms and exact reasons; framework skips remain failures.
  • Extend the existing receipt verifier across Resource, L2, parent execution, xdist and bounded poison retries. Recovered cases count once and an earlier failure cannot be hidden by retry success.
  • Reverify original JUnit before publishing Markdown and machine-readable summaries. Report selected, body-entered, passed, failed, skipped and missing counts with source, environment identity, invocation scope and evidence links. Failed partitions retain attributable partial counts without becoming qualified; invalid plans have no attributable counts. Preserve raw plans, receipts, retry manifests and native capture logs.

Most scene-test/example changes only add metadata; they do not change test bodies or parameter values. No separate capture job or new automatic A5 PR scheduling is introduced.

Validation

  • 238 related CPU regression tests passed, including actual pytest subprocess dispatch, xdist receipts, retries, setup/call/teardown failures, duplicate and foreign receipts, missing shards, strict skips, environment selection, workflow command quoting and Node-executed A5 trigger controls.
  • Three additional collect-only regressions passed for the actual native-submission modules on A2/A3 and A5 and the A5 Host-domain example. They failed before their missing declarations were added. The failure-summary regression was also reproduced against the original implementation.
  • Added three migration-compatibility regressions, reproduced their missing-path failures, then passed all 77 setup/identity/framework/A5-caller tests. The focused suites cover 244 distinct tests in total.
  • Ruff 0.14.8, Pyright 1.1.407, actionlint 1.7.12 (shellcheck/pyflakes integrations disabled), Markdown checks, changed-file header/English/retired-name checks and diff whitespace checks passed. These are focused checks, not a claim of a new full pre-commit or hardware run.
  • Rebased on main ea173476a623533246f7f3c16ec967a4b1fa7b9c; the PR remains one commit.
  • Inspected the original CI run 42193166888: retained receipts at its tested merge source fe215517c955ccc8fff367c99ac9b77695eb80a5 report 15 native HBG capture passes and 24 framework eager passes per runtime. This is evidence for the previous revision, not qualification of the updated head.

Current-head CI evidence

Run 42559820386 tests PR head 04a921c22b33e99440a4ac05cb269d7cc2561b44 at merge source 8fb23b3153e7fd5046cdcac54f172a44ac53d364.

  • The renamed framework job passed. Its uploaded plan, JUnit and summary qualify 30 HBG eager and 28 TMR eager cases, with zero failures/skips/missing results. Source and environment identity are retained (CANN 9.0.0, torch 2.7.1+cpu, torch_npu 2.7.1.post4).
  • Pre-commit, Python/C++ UT, A3 hardware UT, both simulation suites, packaging, profiling smoke and DeepSeek smoke passed. Network1 was still running at the last check.
  • Default onboard is not qualified. Its main report records 268 selected cases, including 21 HBG capture cases: 19 passed and 2 failed. Scenarios 12 and 13 return 507011 on the first injected failure and 507046 on the second; device logs include HandleTaskTimeout. Scenario 13 matches the existing #2923 symptom. This PR changes no runtime/platform implementation; the root-cause repair remains separate. SDMA and fault partitions were not reached after the main step failed and are correctly marked incomplete.
  • Downloaded artifacts confirm the failure report's per-group counts conserve selected cases and retain attributable partial evidence. Framework inventory comparison preserves all 48 previous cases and adds the 10 native-submission cases introduced on main. Default inventory changes from 264 to 268 through six capture cases added on main and two FIFO cases removed on main; the metadata migration did not remove those cases.
  • An attempted single-job rerun was rejected by GitHub with 403: The workflow run containing this job is already running. No rerun has started. Preserve the first-attempt evidence; a later passing retry would be a non-reproduction, not proof that [Bug] HBG capture second boot-failure replay times out in A3 CI #2923 is fixed.

Remaining acceptance

Keep the PR in draft: default onboard acceptance remains blocked by the observed second-fault replay failures, and SDMA partitions still need execution. The renamed framework job and its uploaded evidence have been verified at the current head. Local regression tests use CPU stand-ins and do not qualify hardware execution.

Administrators must confirm any external required-check migration and trigger A5 through the approved /run-a5 policy. Public main rulesets currently show no required status-check rule; the authenticated branch-protection endpoint returned 404 to this write-level account, so legacy protection has not been independently verified or modified. The migration procedure is documented in docs/ci.md.

Daily/manual scope and A5 require their own real execution evidence. Torch dtype-dependent MXFP cases are generated before collection and inventories must be compared under the actual runner dependencies. Unsupported A5/TMR/framework capture combinations remain explicit gaps. These summaries do not claim simulation, Network1, standalone model or DFX-smoke coverage, and this PR does not by itself establish complete issue acceptance.

@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 129 files, which is 29 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Check out review usage here.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6be6fd0e-5f14-4fc2-b8a7-3ccc78788306
Commits

Reviewing files that changed from the base of the PR and between 18e1841 and 7b5f08e.

Files selected for processing: 129

  • .github/actions/setup-framework-eager/action.yml
  • .github/actions/setup-framework-eager/identity.py
  • .github/actions/setup-framework/action.yml
  • .github/actions/setup-framework/constraints.txt
  • .github/actions/setup-framework/identity.py
  • .github/workflows/_st-framework.yml
  • .github/workflows/_st-npu-a2a3.yml
  • .github/workflows/_st-npu-a5.yml
  • .github/workflows/ci-a5-comment.yml
  • .github/workflows/ci-self-comment.yml
  • .github/workflows/ci-self.yml
  • .github/workflows/ci.yml
  • .github/workflows/daily.yml
  • conftest.py
  • docs/ci.md
  • examples/a2a3/host_build_graph/deepseek_v4_flash_decode/test_deepseek_v4_flash_decode.py
  • examples/a2a3/host_build_graph/qwen3_14b_decode/test_qwen3_14b_decode.py
  • examples/a2a3/tensormap_and_ringbuffer/deepseek_v4_flash_decode/test_deepseek_v4_flash_decode.py
  • examples/a2a3/tensormap_and_ringbuffer/qwen3_14b_decode/test_qwen3_14b_decode.py
  • examples/a5/host_build_graph/deepseek_v4_pro_csa/test_deepseek_v4_pro_csa.py
  • examples/a5/host_build_graph/qwen3_14b_decode/test_qwen3_14b_decode.py
  • examples/a5/tensormap_and_ringbuffer/host_comm_domain/test_host_comm_domain.py
  • examples/a5/tensormap_and_ringbuffer/qwen3_14b_decode/test_qwen3_14b_decode.py
  • examples/a5/tensormap_and_ringbuffer/rdma_deferred_completion_demo/test_rdma_deferred_completion_demo.py
  • examples/workers/l2/framework_eager/README.md
  • examples/workers/l2/hello_worker/test_hello_worker.py
  • examples/workers/l2/per_task_runtime_env/test_per_task_runtime_env.py
  • examples/workers/l2/vector_add/test_run_timing.py
  • examples/workers/l2/vector_add/test_vector_add.py
  • examples/workers/l2/worker_malloc/test_worker_malloc.py
  • examples/workers/l3/allreduce/test_allreduce.py
  • examples/workers/l3/child_memory/test_child_memory.py
  • examples/workers/l3/device_copy_offset/test_device_copy_offset.py
  • examples/workers/l3/domain_rank_map/test_domain_rank_map.py
  • examples/workers/l3/dual_domain_overlap/test_dual_domain_overlap.py
  • examples/workers/l3/ep_dispatch_combine/test_ep_dispatch_combine.py
  • examples/workers/l3/ffn_tp_parallel/test_ffn_tp_parallel.py
  • examples/workers/l3/multi_chip_dispatch/test_multi_chip_dispatch.py
  • examples/workers/l3/per_task_runtime_env/test_per_task_runtime_env.py
  • examples/workers/l3/worker_chip_message_queue/test_worker_chip_message_queue.py
  • examples/workers/l3/worker_chip_orch_comm_stream/test_worker_chip_orch_comm_stream.py
  • examples/workers/l4/compute_then_tload_mixed_l3/test_compute_then_tload_mixed_l3.py
  • examples/workers/l4/global_tload_mixed_l3/test_global_tload_mixed_l3.py
  • examples/workers/l4/global_tload_mpirun_l3/test_global_tload_mpirun_l3.py
  • examples/workers/l4/vector_add_mixed_l3/test_vector_add_mixed_l3.py
  • examples/workers/l4/vector_add_mpi_direct_l3/test_vector_add_mpi_direct_l3.py
  • simpler_setup/scene_test.py
  • simpler_setup/test_coverage.py
  • simpler_setup/test_environments.py
  • simpler_setup/tools/README.md
  • simpler_setup/tools/collection_inventory.py
  • simpler_setup/tools/feature_coverage_report.py
  • tests/st/a2a3/host_build_graph/dfx/pmu/test_pmu_counting_window.py
  • tests/st/a2a3/host_build_graph/early_enqueue/test_early_enqueue.py
  • tests/st/a2a3/host_build_graph/prepared_callable/test_prepared_callable.py
  • tests/st/a2a3/kernel_capture/test_hbg_capture.py
  • tests/st/a2a3/kernel_capture/test_kernel_capture.py
  • tests/st/a2a3/tensormap_and_ringbuffer/dynamic_register/test_dynamic_register.py
  • tests/st/a2a3/tensormap_and_ringbuffer/early_enqueue/test_early_enqueue.py
  • tests/st/a2a3/tensormap_and_ringbuffer/l4_network1/test_domain_tensor_network1.py
  • tests/st/a2a3/tensormap_and_ringbuffer/l4_network1/test_global_tload_mixed_l3_network1.py
  • tests/st/a2a3/tensormap_and_ringbuffer/test_prewarm_config.py
  • tests/st/a5/host_build_graph/async_completion/test_condition_boundaries.py
  • tests/st/a5/host_build_graph/async_completion/test_consumer_visibility.py
  • tests/st/a5/host_build_graph/async_completion/test_dependency_completion.py
  • tests/st/a5/host_build_graph/async_completion/test_oracle_negative_control.py
  • tests/st/a5/host_build_graph/async_completion/test_pending_close.py
  • tests/st/a5/host_build_graph/async_completion/test_run_reuse.py
  • tests/st/a5/host_build_graph/async_completion/test_sdma_completion.py
  • tests/st/a5/host_build_graph/async_completion/test_waiter_progress.py
  • tests/st/a5/host_build_graph/async_peer_counter/test_peer_counter.py
  • tests/st/a5/host_build_graph/early_enqueue/test_early_enqueue.py
  • tests/st/a5/host_build_graph/fanout_imbalance/test_fanout_imbalance.py
  • tests/st/a5/host_build_graph/mix_coresidency/test_mix_coresidency.py
  • tests/st/a5/host_build_graph/mix_predicated_dispatch/test_mix_predicated_dispatch.py
  • tests/st/a5/host_build_graph/prepared_callable/test_prepared_callable.py
  • tests/st/a5/host_build_graph/resident_failure/test_resident_failure.py
  • tests/st/a5/host_build_graph/sdma_defer_wait/test_error_close.py
  • tests/st/a5/host_build_graph/sdma_queue_reuse/test_sdma_queue_reuse.py
  • tests/st/a5/host_build_graph/signal_payload_defer/test_pending_close.py
  • tests/st/a5/host_build_graph/spmd_cohort_aiv_full_width/test_spmd_cohort_aiv_full_width.py
  • tests/st/a5/host_build_graph/spmd_cohort_full_width/test_spmd_cohort_full_width.py
  • tests/st/a5/host_build_graph/spmd_cohort_saturation/test_spmd_cohort_saturation.py
  • tests/st/a5/host_build_graph/spmd_cohort_too_wide/test_spmd_cohort_too_wide.py
  • tests/st/a5/host_build_graph/spmd_cohort_with_ordinary/test_spmd_cohort_with_ordinary.py
  • tests/st/a5/host_build_graph/spmd_multiblock_aic/test_spmd_multiblock_aic.py
  • tests/st/a5/host_build_graph/spmd_predicated/test_spmd_predicated.py
  • tests/st/a5/host_build_graph/spmd_sync_start_mix_aiv/test_spmd_sync_start_mix_aiv.py
  • tests/st/a5/host_build_graph/sync_start_capacity/test_sync_start_capacity.py
  • tests/st/a5/tensormap_and_ringbuffer/dynamic_register/test_dynamic_register.py
  • tests/st/a5/tensormap_and_ringbuffer/early_enqueue/test_early_enqueue.py
  • tests/st/a5/tensormap_and_ringbuffer/test_prewarm_config.py
  • tests/st/a5/tensormap_and_ringbuffer/urma_deferred_completion/test_urma_deferred_completion.py
  • tests/st/a5/tensormap_and_ringbuffer/urma_deferred_completion/test_urma_domain_lifecycle.py
  • tests/st/aicore_op_timeout/test_aicore_op_timeout.py
  • tests/st/common/kernel_eager_state/test_stateful_calls.py
  • tests/st/host_build_graph_readiness/test_host_readiness.py
  • tests/st/host_build_graph_validation/test_host_build_graph_validation.py
  • tests/st/run_retention/test_run_retention.py
  • tests/st/runtime_fatal_codes/test_runtime_fatal_codes.py
  • tests/st/task_timing/task_timing_slots/test_task_timing_e2e.py
  • tests/st/vis_isolation/test_vis_isolation.py
  • tests/st/worker/comm_domain/test_domain_tensor_remote.py
  • tests/st/worker/comm_domain/test_unified_domain.py
  • tests/st/worker/comm_region/recursive_single_owner/test_fault_injection.py
  • tests/st/worker/comm_region/recursive_single_owner/test_l3_single_hop.py
  • tests/st/worker/comm_region/recursive_single_owner/test_l4_two_hop.py
  • tests/st/worker/comm_region/templates/spsc_queue/test_failure.py
  • tests/st/worker/comm_region/templates/spsc_queue/test_l3_single_hop.py
  • tests/st/worker/framework_eager/test_alias_inout.py
  • tests/st/worker/framework_eager/test_argument_capture.py
  • tests/st/worker/framework_eager/test_cross_stream.py
  • tests/st/worker/framework_eager/test_diagnostic_selection.py
  • tests/st/worker/framework_eager/test_failure_recovery.py
  • tests/st/worker/framework_eager/test_framework_eager.py
  • tests/st/worker/framework_eager/test_lifecycle.py
  • tests/st/worker/framework_eager/test_native_capture.py
  • tests/st/worker/framework_eager/test_native_submission.py
  • tests/st/worker/framework_eager/test_persistent_state.py
  • tests/st/worker/framework_eager/test_preflight_recovery.py
  • tests/st/worker/framework_eager/test_scope_statistics.py
  • tests/st/worker/framework_eager/test_value_arguments.py
  • tests/ut/py/test_a5_comment_ci.py
  • tests/ut/py/test_dispatch_receipts_prototype.py
  • tests/ut/py/test_feature_coverage_ci.py
  • tests/ut/py/test_feature_coverage_prototype.py
  • tests/ut/py/test_framework_eager_ci_platform.py
  • tests/ut/py/test_framework_eager_contract.py
  • tests/ut/py/test_test_environments.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@zxx0288
zxx0288 force-pushed the support/verify-onboard-feature-coverage branch 5 times, most recently from 3cac638 to 473da4e Compare November 9, 2026 12:13
@zxx0288
zxx0288 marked this pull request as ready for review November 9, 2026 12:34
@hw-native-sys-bot

Copy link
Copy Markdown
Collaborator

@doraemonmj Requested your review: this adds source-bound execution receipts and coverage reports to the onboard environment jobs; 13 pass / 2 skip. Note the PR is currently CONFLICTING, so the author needs to rebase before merge.

@zxx0288
zxx0288 force-pushed the support/verify-onboard-feature-coverage branch from 473da4e to 495e41d Compare November 10, 2026 03:22
@doraemonmj

Copy link
Copy Markdown
Collaborator

Thanks for this — a thorough, well-tested change. I reviewed the full diff (merge-base d24e999 → head 473da4e). No must-fix correctness issues found: the plan→receipt→summary verifier is sound and the CPU regression coverage (real pytest subprocess dispatch + negative controls) is strong. A few points:

Should fix

  • ci-self-comment.yml gate-and-resolve calls core.setOutput('head_sha', …) / core.setOutput('base_sha', …), but neither is declared in the job's outputs: block nor consumed by the lane job. ci-a5-comment.yml declares and consumes both. Mirror that (or drop the two setOutput lines) so the two comment lanes don't drift.

Consider

  • .github/workflows/_st-framework.yml:90 — the artifact-exclusion glob still uses the pre-rename name framework-eager-module*. Harmless today (matches nothing), but it leaves git grep framework-eager dirty.
  • The runtime_product("program") + execution_mode("eager") markers are added to ~80 files that merely restate the default. A global "program/eager" fallback (with only kernel / capture_replay / auxiliary / skip declared explicitly) would cut most of this churn; the tradeoff is a small loss of explicitness/auditability. Not blocking — the size is mostly mechanical — just flagging that the default could absorb the boring case.

Not yet demonstrated on hardware

  • The sdma and sdma_fault partitions haven't run on real silicon (the main step failed first on [Bug] HBG capture second boot-failure replay times out in A3 CI #2923). If any SDMA/fault test lacks a runtime_product/execution_mode marker, remember_candidates → features() raises at collection and that partition reddens on its first real run. Worth running collection_inventory against those two partitions before merge.

Agree with keeping it draft until default-onboard + SDMA acceptance complete.

Classify scene tests by product, runtime and execution mode, and verify
mandatory selected cases against source-bound parent and child receipts.
Preserve device dispatch, exact poison retries and scoped skip exceptions,
including existing quarantines and newly collected native consumers.

Enable verification in existing default and torch_npu onboard workflows.
Name the framework suite by environment across PR, Daily and comment
callers, including setup actions, artifacts and documented check migration.
Preserve triggers, device leases and SDMA isolation.
Retain delegating setup and identity entry points for comment workflows
loaded from the default branch while the migration is pending.
Pin comment-triggered execution to a verified head/base merge snapshot,
so older PR heads consume matching CI interfaces. Reject unavailable or
stale merge snapshots before admitting any suite and report all three SHAs.

Publish per-feature selected, body-entered, passed, failed, skipped and
missing counts with source/environment identity and links to evidence.
Keep partial failure evidence without qualifying missing or invalid
receipts; failures cannot be hidden by retries or cached green summaries.

Keep toolchain bootstrap commands usable before SDK installation and
cover their standalone entry points without site-packages.
Preserve policy-error inventories as JSON with a nonzero exit, and scope
the Daily counting-window PMU skip to its exact platform and reason.

Related to hw-native-sys#2904
@zxx0288
zxx0288 force-pushed the support/verify-onboard-feature-coverage branch from 495e41d to 7b5f08e Compare November 10, 2026 05:18
@zxx0288

zxx0288 commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

@doraemonmj Thanks for the review. Addressed the unused-output finding: removed the head_sha and base_sha step outputs from /run-self; the gate Job Summary still records the PR head, base, and tested merge SHA.

Added an assertion that emitted gate outputs match the job's declared outputs. All 46 related comment-workflow tests, Ruff, and actionlint passed. These are focused validation results, not a claim that the updated head's full CI has completed.

@hw-native-sys-bot

Copy link
Copy Markdown
Collaborator

@doraemonmj Re-review requested on head 7b5f08e4: CI is green (13 pass / 2 skip) and the PR is MERGEABLE. @zxx0288 addressed the unused head_sha/base_sha outputs and added an assertion that emitted gate outputs match the declared job outputs. The remaining Consider items and the SDMA/collection check are for the author to confirm.

@doraemonmj

Copy link
Copy Markdown
Collaborator

/run-a5

1 similar comment
@ChaoZheng109

Copy link
Copy Markdown
Collaborator

/run-a5

@github-actions

Copy link
Copy Markdown

/run-a5 finished without all suites passing

Tested head: zxx0288/simpler@7b5f08e49ed43ba0b969848fcb4dccb2a9794534
Workflow run

  • ut-a5: failure
  • st-onboard-a5: failure
  • st-framework-eager-a5: success

@ChaoZheng109

Copy link
Copy Markdown
Collaborator

/run-a5

@github-actions

Copy link
Copy Markdown

/run-a5 finished without all suites passing

Tested head: zxx0288/simpler@7b5f08e49ed43ba0b969848fcb4dccb2a9794534
Workflow run

  • ut-a5: failure
  • st-onboard-a5: failure
  • st-framework-eager-a5: success

@ChaoWao

ChaoWao commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Two suggestions to keep the coverage check maintainable:

  1. Could we make execution_mode a Kernel-only field? Group Program coverage as program/<runtime>. Group Kernel coverage as kernel/<runtime>/eager or kernel/<runtime>/capture_replay. Within the existing ST coverage scope, classify the exceptions and default ordinary cases to Program. Declare Kernel once in the shared framework test setup and Kernel/capture_replay once for the native capture suite. The framework suite covers both modes, so declare mode at the scope that shares it. In test_stateful_calls.py, tag the existing borrowed parameter and the borrowed-only case as Kernel/eager. The managed parameter can use the Program default. Keep auxiliary probes explicit. Use the same group keys in the coverage report and required groups.

  2. Could the workflow pass the coverage partition explicitly? _partition() recognizes specific spellings of the pytest -m expression. For example, not sdma_fault and not sdma selects the same tests as not sdma and not sdma_fault, but the former falls back to the all coverage partition. An explicit partition would let pytest handle selection while the verifier checks the intended coverage.

@hw-native-sys-bot

hw-native-sys-bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

@zxx0288 The PR is now CONFLICTING after the recent main moves; please rebase onto current main so doraemonmj can re-review the latest head. CI was green before the conflict.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants