From 666087def880905ce46db6be926e00fef73d6135 Mon Sep 17 00:00:00 2001 From: Javier Tia Date: Tue, 29 Sep 2026 13:34:35 -0600 Subject: [PATCH] imx8mp-npu-pose: Report Vela op placement at build time Nothing in the build says which ops stay on the CPU. quantize-model.py only catches a degenerate quantization, so a model that converts cleanly but falls back to the CPU shows up as low FPS on the board. On a target that uses Vela, its per-op report is thrown away because nothing runs it. Add check-op-placement.sh and call it from app-compile.sh after quantization. Where Vela is in the SDK it compiles the INT8 model for the Ethos-U65, prints the CPU vs NPU operator counts and the CPU ops, and fails when the CPU share is over MAX_CPU_PCT. Without Vela it prints a skip line, so the 8M Plus build is unchanged. The default limit is 15%. Vela 5.2.0 puts 29 of this model's 221 ops (13.1%) on the CPU, all of them decode and quantize-boundary ops, so the limit guards against regressions above that baseline rather than setting a target. MAX_CPU_PCT must be a number, because awk compares a non-number as a string and "5%" would pass any model. No target enables the check yet. A target turns it on by adding nativesdk-ethos-u-vela through a target-: override. Signed-off-by: Javier Tia --- imx8mp-npu-pose/README.md | 6 ++++ imx8mp-npu-pose/app-compile.sh | 3 ++ imx8mp-npu-pose/check-op-placement.sh | 51 +++++++++++++++++++++++++++ 3 files changed, 60 insertions(+) create mode 100755 imx8mp-npu-pose/check-op-placement.sh diff --git a/imx8mp-npu-pose/README.md b/imx8mp-npu-pose/README.md index c5dad8a..cb6df38 100644 --- a/imx8mp-npu-pose/README.md +++ b/imx8mp-npu-pose/README.md @@ -70,6 +70,12 @@ to pixels (no letterbox math). tail has a couple of reduce/gather ops that may land on CPU — negligible, but if the graph fragments badly, PoseNet (pure-conv heatmaps + NNStreamer's built-in `mode=pose_estimation` decoder) is the drop-in fallback. + The 8M Plus has no build-time check for this. Where a target's SDK includes + Vela (`nativesdk-ethos-u-vela`, added through a `target-:` override), + `app-compile.sh` also runs `check-op-placement.sh`, which prints Vela's CPU vs + NPU operator counts and the CPU ops, and fails the build when the CPU share + passes `MAX_CPU_PCT` (default 15). Vela 5.2.0 puts 29 of this model's 221 + operators (13.1%) on the CPU, all of them decode and quantize boundary ops. - **uint8 input:** if a MoveNet export resists `inference_input_type=uint8` (its SavedModel input is int32), see `quantize-model.py` — wrap with a float input layer or fall back to PoseNet. diff --git a/imx8mp-npu-pose/app-compile.sh b/imx8mp-npu-pose/app-compile.sh index 39d556a..9ef234c 100755 --- a/imx8mp-npu-pose/app-compile.sh +++ b/imx8mp-npu-pose/app-compile.sh @@ -43,6 +43,9 @@ fi "$QPY" quantize-model.py "$MODEL_DIR" "$REP_DIR" "$OUT_DIR/movenet_int8.tflite" +# Reports the CPU/NPU op split where Vela is in the SDK (i.MX93); skipped elsewhere. +./check-op-placement.sh "$OUT_DIR/movenet_int8.tflite" + echo "" echo "Quantization complete:" ls -lh "$OUT_DIR/" diff --git a/imx8mp-npu-pose/check-op-placement.sh b/imx8mp-npu-pose/check-op-placement.sh new file mode 100755 index 0000000..e5eaa68 --- /dev/null +++ b/imx8mp-npu-pose/check-op-placement.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# +# Report which ops Vela leaves on the CPU for the INT8 model, and fail the build +# when the CPU share of operators is over a limit. Vela only exists in the SDK on +# targets that add it through a target-: override in avocado.yaml, so this +# is a no-op everywhere else (the 8M Plus runs through the VX delegate, which has +# no Vela step). +# +# MoveNet's decode (ArgMax, GatherNd, Sigmoid, casts and the quantize/dequantize +# boundaries) stays on the CPU by design. Measured with Vela 5.2.0 on this +# model: 29 CPU operators of 221 (13.1%). The default limit is a regression guard +# just above that, not a target; raise it with MAX_CPU_PCT if the model changes. + +set -euo pipefail + +MODEL="${1:?usage: check-op-placement.sh }" +MAX_CPU_PCT="${MAX_CPU_PCT:-15}" +OUT_DIR="$(dirname "$MODEL")/vela" +SUMMARY="$OUT_DIR/vela-summary.txt" + +if ! command -v vela >/dev/null 2>&1; then + echo "vela not in this SDK; skipping op-placement report" + exit 0 +fi + +# awk falls back to string comparison on a non-number (5% would never fail). +if ! [[ "$MAX_CPU_PCT" =~ ^[0-9]+(\.[0-9]+)?$ ]]; then + echo "ERROR: MAX_CPU_PCT must be a number, got '$MAX_CPU_PCT'" >&2 + exit 1 +fi + +mkdir -p "$OUT_DIR" +vela --accelerator-config ethos-u65-256 --show-cpu-operations \ + --output-dir "$OUT_DIR" "$MODEL" >"$SUMMARY" + +# The full per-op listing (NPU ops included) stays in $SUMMARY; the build log +# gets the counts and the CPU ops only. +grep -E '^(CPU|NPU) operators = |^ CPU: ' "$SUMMARY" || true + +cpu_line="$(grep -E '^CPU operators = ' "$SUMMARY")" || { + echo "ERROR: no operator summary in Vela output; see $SUMMARY" >&2 + exit 1 +} +cpu_pct="$(sed -E 's/.*\( *([0-9.]+)%\).*/\1/' <<<"$cpu_line")" + +echo "" +echo "Op placement: $cpu_line (limit ${MAX_CPU_PCT}%)" +if awk -v p="$cpu_pct" -v m="$MAX_CPU_PCT" 'BEGIN { exit !(p > m) }'; then + echo "ERROR: CPU op share ${cpu_pct}% is over ${MAX_CPU_PCT}%; see $SUMMARY" >&2 + exit 1 +fi