Wagtail team reports 50% failure rate in one-month GLM 5.3 Flash challenge

Key hurdles included the high cost of "vibe coding" a prototype for their Wagtail MCP server, which consumed 450M tokens and $150 almost overnight due to poor model selection. Additionally, the team faced infrastructure availability issues, noting performance degradation with GLM 5.3 Flash because of high demand on inference providers. This forced them to switch to alternative models like DeepSeek V4.1 Flash and Qwen 3.8 Flash. The team also emphasized the need to continue experimenting with a wide range of models to benchmark performance on specific tasks.

Despite the failure, the team identified several lessons for October, including the need for constant local usage measurement, better budgeting for experimentation, and improved prompt selection using multi-agent techniques. They concluded that while focusing on one or two efficient flash-tier models is viable for day-to-day work, the majority of AI inference should be measured by cost or energy use rather than token count. The team plans to share these findings at Wagtail Space 2026 in November.