Open-source panel lets users run llama.cpp, stable-diffusion.cpp and audio.cpp locally

The UI lets users create any number of inference instances, each pinned to a chosen GPU or CPU, with independent parameters, ports and logs. Over 200 settings are grouped with plain‑English tooltips, and a VRAM/RAM estimator refuses launches that would exceed available memory. Features include fallback tiers that downgrade configurations after repeated failures, an optimizer that benchmarks settings against coding and agent workloads, and a decode‑vs‑depth curve that measures tokens per second at various context lengths while tracking GPU temperature. Models can be downloaded directly from Hugging Face, and the panel offers an in‑browser terminal, file manager, and detailed crash forensics.

The panel runs on Ubuntu 26.04 with Python 3.14, Caddy 2.x for TLS and basic auth, and ttyd for the terminal. Reference hardware in the documentation is an AMD Radeon 7900 XTX, but the setup also supports NVIDIA via nvidia‑smi and Vulkan‑NVK. All configuration is stored in plain files, making backup and migration straightforward.