Developing story · 2 episodes · 27 Sept to 29 Sept
Local AI Inference Speeds Hit New Peaks
The saga tracks the rapid advancement of local AI inference performance on consumer hardware, highlighting significant speedups in prompt processing and token generation. The latest development confirms that optimized models now achieve near-real-time speeds on Apple's newest silicon, marking a major milestone for on-device AI capabilities.
-
Swift-1.5-Qwen3.8-27b-oQ8e-mtp achieves 34.8 tokens per second on Apple M5 Max
A Reddit user shared benchmark results for the Swift-1.5-Qwen3.8-27b-oQ8e-mtp model on an Apple M5 Max device on September 29, 2026. The model reached 34.8 tokens per second in a coding scenario,…
1 source -
Llama.cpp achieves up to 42x faster prompt lookup drafting
Benchmarks were run on an Apple M4 Pro (14 cores, 48 GB RAM) using the WikiText‑103 corpus (541 MB) and smaller subsets (25 MB‑200 MB). Median latency per drafted token and memory figures were…
1 source
Who and what