diff --git a/README.md b/README.md index 54bcdf6d..64c8fb69 100644 --- a/README.md +++ b/README.md @@ -176,13 +176,18 @@ mlops: model: clayer: $AI-Layer$ name: TinyCNN + input-shape: 1x16x16x3 + calibration-samples: 2 ``` `cbuild setup --active SSE-320-U85` resolves it into `cmsis-executorch.cbuild-mlops.yml`, which contains the processor, NPU -and Vela options. `create_ai_layer.py` reads those options and passes all of -them to ExecuTorch's `EthosUCompileSpec`, so the target configuration is never -duplicated in Python. The script then writes: +and Vela options, and the FVP target-set as the `simulator:` to test on. +`create_ai_layer.py` reads those options and passes all of them to ExecuTorch's +`EthosUCompileSpec`, so the target configuration is never duplicated in +Python. The two extra keys under `model:` are parameters of the model itself; +the toolbox passes them through and the script hands them to `model/model.py`. +The script then writes: - `ai_layer/ai_layer.clayer.yml`: the CMSIS components required by the model. - `ai_layer/model_pte.c` and `model_pte.h`: the ExecuTorch program embedded as a C array. @@ -202,8 +207,8 @@ changes the operator set needs nothing more than re-running steps 2 and 3. ## Adapting the example To use a different model, replace or modify `model/model.py` and update the -model name or input handling as required: `INPUT_SHAPE` and -`get_model(input_shape)` define the input, and +model name or input handling as required: `get_model(input_shape)` builds the +model for the `input-shape` of the `mlops:` node, and `get_calibration_inputs(input_shape, calibration_samples)` returns the samples the quantizer is calibrated with; give it representative data for a trained model. The runner in `src/app_main.cpp` builds its input tensor with the same diff --git a/cmsis-executorch.csolution.yml b/cmsis-executorch.csolution.yml index 89a55385..ba2d260b 100644 --- a/cmsis-executorch.csolution.yml +++ b/cmsis-executorch.csolution.yml @@ -59,17 +59,21 @@ solution: # MLOps information: exported to cmsis-executorch.cbuild-mlops.yml by # `cbuild setup ... --active SSE-320-U85`. Retarget the NPU here and - # create_ai_layer.py picks it up on its next run. + # create_ai_layer.py picks it up on its next run. The simulator target comes + # from the Arm-FVP target-set, and the extra keys under model: are passed + # through to create_ai_layer.py. # https://open-cmsis-pack.github.io/cmsis-toolbox/build-overview/#mlops-information mlops: description: TinyCNN int8 image classifier for Ethos-U85 npu: - type: Ethos-U85 + type: Ethos-U85 # explicit: the SSE-320 device pack declares no NPU vela: system: Ethos_U85_SYS_DRAM_Mid # system-config from the Vela config memory: Shared_Sram # memory-mode from the Vela config model: clayer: $AI-Layer$ name: TinyCNN - simulator: - target: SSE-320-U85 + # Model parameters for create_ai_layer.py: the NHWC input shape and the + # number of samples the quantizer is calibrated with. + input-shape: 1x16x16x3 + calibration-samples: 2 diff --git a/create_ai_layer.py b/create_ai_layer.py index aeab3392..e55cce24 100644 --- a/create_ai_layer.py +++ b/create_ai_layer.py @@ -15,7 +15,9 @@ of the csolution. This script reads the NPU and Vela settings from it, exports the PyTorch model in model/model.py for that NPU (quantize, delegate to Ethos-U, compile with Vela) and writes the complete AI layer into the -directory of the clayer named under `model.clayer`: +directory of the clayer named under `model.clayer`. The extra keys of the +`model:` node (input-shape, calibration-samples) are model parameters that +CMSIS-Toolbox passes through to this script: ai_layer.clayer.yml runtime, kernel utilities and registration, Ethos-U backend and the operator components the exported @@ -164,7 +166,22 @@ def compile_spec(mlops: dict, mlops_dir: Path): return EthosUCompileSpec(**kwargs) -def export_model(spec) -> bytes: +def model_params(mlops: dict) -> dict: + """The extra keys of the model: node, as keyword arguments for model/model.py.""" + params = {} + for key, value in mlops["model"].items(): + if key == "input-shape": + params["input_shape"] = tuple(int(d) for d in re.split(r"[x,]", str(value))) + elif key == "calibration-samples": + params["calibration_samples"] = int(value) + elif key not in ("clayer", "name"): + print(f"[ai_layer] warning: ignoring unknown model key {key!r}", file=sys.stderr) + if params: + print(f"[ai_layer] model parameters: {params}") + return params + + +def export_model(spec, params: dict) -> bytes: """Quantize model/model.py, delegate it to the Ethos-U and return the .pte.""" import torch from executorch.backends.arm.ethosu import EthosUPartitioner @@ -175,7 +192,8 @@ def export_model(spec) -> bytes: sys.path.insert(0, str(HERE / "model")) from model import get_calibration_inputs, get_model - model, samples = get_model(), get_calibration_inputs() + model = get_model(**{k: v for k, v in params.items() if k == "input_shape"}) + samples = get_calibration_inputs(**params) example = (samples[0],) graph = torch.export.export(model, example).module() @@ -293,7 +311,7 @@ def main() -> None: layer_file = mlops_file.parent / mlops["model"]["clayer"] layer_dir = layer_file.parent - pte = export_model(compile_spec(mlops, mlops_file.parent)) + pte = export_model(compile_spec(mlops, mlops_file.parent), model_params(mlops)) version = executorch_version(mlops_file) runtime, operators = components(pte, executorch_pack(version)) diff --git a/documentation/mlops-flow.md b/documentation/mlops-flow.md index dd46e9a3..ed93f9bb 100644 --- a/documentation/mlops-flow.md +++ b/documentation/mlops-flow.md @@ -34,17 +34,21 @@ solution: model: clayer: $AI-Layer$ name: TinyCNN - simulator: - target: SSE-320-U85 # [@] of the FVP + input-shape: 1x16x16x3 # passed through to create_ai_layer.py + calibration-samples: 2 ``` +CMSIS-Toolbox detects the `simulator:` target (and, for a board, the +`hardware:` target) from the target-sets, and passes the extra keys under +`model:` through unchanged. + `cbuild setup cmsis-executorch.csolution.yml --active SSE-320-U85` resolves it for the active target and writes `cmsis-executorch.cbuild-mlops.yml`: ```yaml cbuild-mlops: - generated-by: csolution version 2.14.1+p38-gf512b381 + generated-by: csolution version 2.15.1+p3-gf46d68bf description: TinyCNN int8 image classifier for Ethos-U85 processor: type: Cortex-M85 @@ -55,6 +59,8 @@ cbuild-mlops: model: clayer: ai_layer/ai_layer.clayer.yml name: TinyCNN + input-shape: 1x16x16x3 + calibration-samples: 2 simulator: active: SSE-320-U85 cbuild-run: out/cmsis-executorch+SSE-320-U85.cbuild-run.yml