ToshLLM adds AMD GPU Metal support for Intel Macs
ToshLLM, a local LLM inference app for Intel Macs with AMD GPUs, now supports Metal acceleration for AMD Polaris (GCN 4) cards like the RX 580. The project, built on patches to llama.cpp, claims near-Vulkan performance on an RX 580 running macOS 12, with benchmarks showing a 66.7% speedup in prefill and 19% in decode over its previous branch. The app includes features like native chat, agent…
Key points
- ToshLLM achieves **66.7% faster prefill** and **19% faster decode** on AMD RX 580 via Metal acceleration patches to `llama.cpp`
- Supports **native chat, agent tools, math/science plugins, GPU speech, and image generation** on Intel Macs with AMD GPUs
- Dynamic MoE memory management and multi-GPU support improve performance for large models like Qwen3.6-35B
RX580(Polaris AMD) on a MAC with near Vulkan performance!
github.com · 3 November 2026Loading the full article…
This text was published by github.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Hardware & Compute
All →- Apple rolls out beta testing for Audio Intelligence features on Apple Watch · 1 src
- OpenAI launches Grid Protectors program for Georgia campus · 3 src
- IO Fund analyst sees $1T AI infrastructure opportunity for server firms · 1 src
- Defeating NVIDIA? The story of how I built a Strata AI server with unpopular Radeon GPUs and had it build a 3D Minecraft game automatically for 140 yen in elect · 1 src
- Innoscience shows GaN power solutions for AI servers at OCP Summit · 1 src
Comments
via GitHub Discussions