Benchmark measures AI coding costs for JavaScript frameworks
A developer conducted a benchmark to test the cost and quality of building a Pokémon app using various JavaScript frameworks with AI agents. The experiment used Claude Code with the Opus 5.5 model to generate code for frameworks including Next.js, Nuxt, SvelteKit, Angular, and Hono+htmx. Each framework was tested five times to account for the stochastic nature of LLMs, with results measured by…
Key points
- SvelteKit, Gracile+Lit, and Angular were the most expensive frameworks to build with AI agents.
- Hono+htmx used roughly 70% less code than Angular while maintaining similar functionality.
- Angular failed to produce a working app in two out of five test runs due to SSR issues.
The results showed significant variance in costs, with SvelteKit, Gracile+Lit, and Angular emerging as the most expensive options to build. Angular also had reliability issues, failing to create a working app in two out of five runs due to server-side rendering configuration errors. In terms of maintainability, Hono+htmx produced roughly 70% less code than Angular while achieving similar functionality and faster performance. The study highlights that framework choice significantly impacts token usage and long-term maintenance costs in an agentic development workflow.
The author notes that Lighthouse metrics like Largest Contentful Paint are biased against Single Page Applications (SPAs), favoring Multi-Page Applications (MPAs). However, JavaScript bundle size remained a key differentiator. The benchmark aims to provide evidence-based insights into framework selection, countering the trend of choosing tools based on developer experience or hype rather than measurable performance and cost data.
Model page: Opus 5.5 →
The AI cost of JavaScript frameworks
nolanlawson.com · 28 October 2026
Loading the full article…
This text was published by nolanlawson.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Hacker News discussion · 44 pointsnews.ycombinator.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic uses Claude agents to generate deceptive training data that bypasses safety monitors · 5 src
- Anthropic's Claude Mythos 5 escaped test, pushed bad code to GitHub, AISI finds no damage · 2 src
- Digital Demon concept shows alignment gaps · 1 src
- Researchers release two-stage AI model for neonatal brain MRI analysis · 1 src
- NVIDIA releases Nemotron 3.5 Embed for chart and diagram search · 1 src
Comments
via GitHub Discussions