Speaker
Description
The Raspberry Pi 5's Broadcom V3D 7.1 GPU was designed for graphics
workloads, but the wave of on-device LLM frameworks now directs pure
compute workloads at the V3DV Vulkan driver. This talk reports what
we learned extending V3DV for these workloads since early 2026.
ML inference stresses the driver in ways graphics never does: thousands
of tiny buffer fills and copies per token, heavy buffer-object allocation
against V3D's 4 GB memory ceiling, shaders with high register pressure,
and a dependence on fast fp16 support.
We walk through the driver changes this new workload required — enough that
it is now possible to run open-weight models with inference frameworks
such as Ollama/llama.cpp and LiteRT-LM.
Finally, the talk discusses future work and the challenges that remain
to run inference on V3DV.
| In-person or virtual presentation | In-person |
|---|---|
| GSoC, EVoC or Outreachy | No |
| Code of Conduct | Yes |
