New Learning Path: Measure whether LLM decode is memory-bound on an Arm CPU

[manunicholasjacob]

New Learning Path: Measure whether LLM decode is memory-bound on an Arm CPU (content/learning-paths/cross-platform/llm-decode-roofline/).

It shows how to measure the memory-read ceiling of an Arm board, run llama.cpp across model sizes and thread counts, and use llama-roofline to check whether decode is limited by memory bandwidth, then pick the thread count for decode and prefill.

What is in the PR

• _index.md, _next-steps.md and 5 pages: how decode uses memory, setup, measure bandwidth, diagnose, threads and size.
• Category cross-platform, shared with embedded, laptops and servers. Advanced, 45 minutes.
• One line in assets/contributors.csv and 6 words in .wordlist.txt.

How it was tested

• Raspberry Pi 5 (4 x Cortex-A76, 2 GB, Raspberry Pi OS, kernel 6.12), llama.cpp 836d571 built in Release with GCC 12.2.0, llama-roofline 0.2.0 from PyPI, official Qwen2.5 GGUF files (0.5B Q4_0, 0.5B Q8_0, 1.5B Q4_0) with sizes and sha256 checked.
• Every output block in the pages comes from one capture run, not illustrative output.
• Results: read ceiling 13.78 GB/s; decode fits 11.82 GB/s divided by bytes per token (R squared 0.9996, 3 models) and runs at 84 to 86% of the ceiling; decode knee at 2 threads, 4 threads lose 19.4%; prefill scales 3.10x from 1 to 4 threads.
• No thermal throttling during the run (get_throttled 0x0 at four points); all sweep rows within 3% CV.
• Only measured on the Raspberry Pi 5; the commands are generic Arm Linux.

Local checks run before opening: full hugo 0.130.0 extended site build (no warnings or errors), htmltest internal links, pyspelling with the repo config, tools/verify_index_fields.py, tools/style_check.py, and a profanity scan. All pass.

Disclosures: I maintain llama-roofline (MIT). AI assistance was used for drafting and running the checks; I ran the measurements and reviewed every page.

Before submitting a pull request for a new Learning Path, please review Create a Learning Path

• I have reviewed Create a Learning Path

Please do not include any confidential information in your contribution. This includes confidential microarchitecture details and unannounced product information.

• I have checked my contribution for confidential information

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the Creative Commons Attribution 4.0 International License.