Repository navigation
[executorch][muse] Fix the DFlash final-block draft counts test - #24050
Merged
Merged
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24050
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
|
|
Gasoonjia
force-pushed
the
gasoonjia/fix-dflash-final-block-test
branch
from
October 21, 2026 17:25
caf7d3a to
6e13047
Compare
Gasoonjia
marked this pull request as ready for review
October 22, 2026 05:56
shoumikhin
approved these changes
Oct 22, 2026
Gasoonjia
added this pull request to stack #24077
October 22, 2026 07:28
Gasoonjia
force-pushed
the
gasoonjia/fix-dflash-final-block-test
branch
from
October 22, 2026 17:55
6e13047 to
7d911c2
Compare
This PR needs a
|
CudaDFlashNative.FinalWholeBlockFitsExactlyAndPreservesAllPrefillRows has failed in unittest-cuda-muse-glimmer-batching since #24043 added draft forward count checks to it: Expected: (drained.draft_forwards - drain.draft_forwards) > (drained.static_draft_forwards - drain.static_draft_forwards), actual: 1 vs 1 The test expected the decode at position 78 to drain the 79 hidden rows dynamically before the static draft. But run_target drains every owner it touches right after appending hidden rows (#24042), so the 78-row prefill already drains down to the fresh context (32). The decode then holds at most one block of rows and takes a single static draft, and the dynamic forwards fall before the window the test measured. Measure from before the prefill instead: the prefill drains through dynamic drafts only, and the decode runs exactly one static draft.
Gasoonjia
force-pushed
the
gasoonjia/fix-dflash-final-block-test
branch
from
October 22, 2026 21:07
7d911c2 to
03e2b91
Compare
Gasoonjia
added a commit
that referenced
this pull request
Oct 22, 2026
…r warnings (#24052) Stacked on #24050. Every CUDA method load that binds FQN weights next to an off-graph KV cache printed one AOTI warning line per KV buffer: ``` [WARNING] Found constant or module state buffer or empty module state parameter __et_offgraph_kv_layer_0_k in model, but not provided by user! ``` The `unittest-cuda-muse-glimmer-batching` log contains 1672 of these lines, out of 11650 lines total. **Why it happens.** `CudaWeightCache::load` binds the weights with `validate_full_update=true` and intentionally leaves the off-graph KV buffers unbound, because the KV pool binds them later. AOTI's full-update check (`assert_all_constants`) allows absent buffers, but it writes a warning to stderr for each absent buffer on every load. **Fix.** - When the AOTI library exports `AOTInductorModelContainerGetConstantType` and `AOTInductorModelContainerGetConstantFromFolded`, ExecuTorch runs the same completeness rule itself. Folded constants, buffers, tensor constants, and empty parameters may be absent. Every other constant must be bound, otherwise `load` fails with `InvalidProgram`. - Binding then runs with `validate_full_update=false`. In AOTI that flag only gates `assert_all_constants`, so nothing else about binding changes. - If the library does not export those two APIs, `load` keeps `validate_full_update=true` as before. Test plan: - `g++ -fsyntax-only` on `cuda_weight_cache.cpp` and `cuda_backend.cpp` against the CUDA 12.6 headers. - CI: the CUDA unittest jobs and the muse-glimmer batching job should pass, and their logs should no longer contain the `__et_offgraph_kv_` warnings.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #24047.
CudaDFlashNative.FinalWholeBlockFitsExactlyAndPreservesAllPrefillRowshas failed inunittest-cuda-muse-glimmer-batchingsince #24043. It failed on that PR's own CI and onmainat 8d9eec3:The test expected the decode at position 78 to drain the 79 hidden rows through the dynamic draft before running the static draft. But
run_targetdrains every owner it touches right after it appends hidden rows (added in #24042). So the 78-row prefill already drains down to the fresh context (32 in this fixture). The decode then holds at most one block (16) of rows and runs a single static draft. The dynamic forwards happen during the prefill, before the stats snapshot the test took.This is a bug in the test's measurement window, not in the executor. The fix takes the first snapshot before the prefill and checks two things:
Test plan:
clang-formatis clean on the changed lines.unittest-cuda-muse-glimmer-batchingon this PR should pass.