# Artificial Analysis releases smartphone inference ranking showing model scores drop under one-minute cutoff

Digest AI · Research · published 2026-09-28T05:07:00Z

Canonical: https://digestai.news/story/artificial-analysis-releases-smartphone-inference-ranking-showing-mode

## Summary

Artificial Analysis published its 'Smartphone Inference Ranking' on August 24, 2026, evaluating small AI models on iPhone 17 Pro and Galaxy S26 Ultra devices. The ranking, developed with Liquid AI, measures intelligence via multiple benchmarks and speed/memory using llama.cpp and GGUF-format models. On iPhone 17 Pro, Nanbeige4.2-3B and LFM2.5-2.6B tied for first at 63 points, but under a one-minute generation cutoff, Nanbeige4.2-3B dropped to 18 points while LFM2.5-8B-A1B led with 47 points. The article explains that standard speed metrics measure fixed 256-token output, not real-world inference which can require thousands of tokens. Measurements were conducted under active cooling and external power, which may not reflect real-world device performance. Liquid AI, which developed the measurement tool Pipette, also supplied several top-ranking models, raising potential conflicts of interest, though they disclosed their involvement and made the methodology open source.

## Key points

- Nanbeige4.2-3B and LFM2.5-2.6B tied at 63 points on iPhone 17 Pro under standard conditions
- Under one-minute cutoff, Nanbeige4.2-3B fell to 18 points, LFM2.5-8B-A1B led with 47 points
- Speed metrics measure 256-token output, not full inference which can exceed 16K tokens

## Why it matters

The ranking reveals that smartphone AI usability depends on real-world inference time, not just benchmark scores, helping developers choose models based on actual user wait limits rather than misleading speed numbers.

## Sources

1. [An AI with a score of 63 drops to 18 in one minute—Reading Artificial Analysis's 'Smartphone Inference Ranking' from primary sources|アイドリ | AI-Driven Lab](https://note.com/ai_driven/n/n27095785d3eb?hl=en) (note.com, 2026-09-28)

## Cite

Digest AI, "Artificial Analysis releases smartphone inference ranking showing model scores drop under one-minute cutoff", 28 September 2026, https://digestai.news/story/artificial-analysis-releases-smartphone-inference-ranking-showing-mode

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/artificial-analysis-releases-smartphone-inference-ranking-showing-mode.json