Skip to content

Report Vela op placement in imx8mp-npu-pose - #36

Merged
jetm merged 1 commit into
mainfrom
pose-op-placement-report
Oct 2, 2026
Merged

jetm merged 1 commit into
mainfrom
pose-op-placement-report

Conversation

@jetm

@jetm jetm commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Problem

Nothing in the build says which ops stay on the CPU. quantize-model.py
only catches a degenerate quantization, so a model that converts cleanly
but falls back to the CPU shows up as low FPS on the board. On a target
that uses Vela, its per-op report is thrown away because nothing runs it.

Solution

check-op-placement.sh runs from app-compile.sh after quantization.
Where Vela is in the SDK it compiles the INT8 model for the Ethos-U65,
prints the CPU vs NPU operator counts and the CPU ops, and fails when the
CPU share is over MAX_CPU_PCT (default 15). Without Vela it prints a skip
line, so the 8M Plus build is unchanged.

Key changes

  • New check-op-placement.sh, called from app-compile.sh.
  • README's "NPU caveats" now describes the check and the measured baseline.
  • No avocado.yaml change. A target turns the check on by adding
    nativesdk-ethos-u-vela through a target-<name>: override.

Reviewer notes

Vela 5.2.0 (PyPI, not the NXP fork) on this model's real INT8 output puts 29
of 221 ops (13.1%) on the CPU, all decode and quantize-boundary ops, so 15% is
a regression guard, not a target. Tested against that Vela: pass, over-limit
fail, no-Vela skip, unparseable-output fail, and a non-numeric limit fail;
shellcheck clean. A local avocado build --target ucm-imx8m-plus runs the hook
(skip line) and builds the app extension, then stops at stone validate
(imx-boot not found). The same failure happens on untouched main on my
machine (CLI 1.0.0-rc.4), so CI's build check is the real gate here.

Feedback wanted: whether an operator-count share is the right signal, or
whether you would rather gate on estimated cycles.

Nothing in the build says which ops stay on the CPU.
quantize-model.py only catches a degenerate quantization, so a model
that converts cleanly but falls back to the CPU shows up as low FPS on
the board. On a target that uses Vela, its per-op report is thrown away
because nothing runs it.

Add check-op-placement.sh and call it from app-compile.sh after
quantization. Where Vela is in the SDK it compiles the INT8 model for
the Ethos-U65, prints the CPU vs NPU operator counts and the CPU ops,
and fails when the CPU share is over MAX_CPU_PCT. Without Vela it
prints a skip line, so the 8M Plus build is unchanged.

The default limit is 15%. Vela 5.2.0 puts 29 of this model's 221 ops
(13.1%) on the CPU, all of them decode and quantize-boundary ops, so
the limit guards against regressions above that baseline rather than
setting a target. MAX_CPU_PCT must be a number, because awk compares a
non-number as a string and "5%" would pass any model.

No target enables the check yet. A target turns it on by adding
nativesdk-ethos-u-vela through a target-<name>: override.

Signed-off-by: Javier Tia <javier@peridio.com>
@github-actions

Copy link
Copy Markdown

Avocado References — Build Check

Reference Target install build Result
imx8mp-npu-pose ucm-imx8m-plus ✅ ✅ PASS

Totals: 1 ✅ passed · 0 ❌ failed

Per-reference logs: workflow run

@jetm
jetm merged commit 1ed9fe4 into main Oct 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants