{"version":1,"type":"story","url":"https://digestai.news/story/neo-classic-benchmark-evaluates-linguistic-aesthetic-reasoning-in-clas","json":"https://digestai.news/story/neo-classic-benchmark-evaluates-linguistic-aesthetic-reasoning-in-clas.json","markdown":"https://digestai.news/story/neo-classic-benchmark-evaluates-linguistic-aesthetic-reasoning-in-clas.md","slug":"neo-classic-benchmark-evaluates-linguistic-aesthetic-reasoning-in-clas","headline":"Neo-Classic benchmark evaluates linguistic-aesthetic reasoning in Classical Chinese poetry","summary":"Researchers present Neo-Classic, a new evaluation benchmark that uses out‑of‑sample, contemporary metrical poems written by experts to probe linguistic‑aesthetic reasoning in large language models. Unlike prior tests that rely on historical corpora, Neo-Classic includes reverse‑understanding probes that reduce the chance of simple retrieval.\n\nThe study assesses three state‑of‑the‑art models—Qwen3-Max, Gemini-3-Pro, and DeepSeek-V3.2—across five behavioral probes. Results show a 20 to 50 percent performance gap when models move from historical to contemporary texts, and discourse‑level ordering tasks yield very low accuracy, ranging from 0 to 13 percent. Providing expert‑level guidance improves reasoning‑enhanced models to 36 percent, yet a sizable gap remains compared with human experts. The findings suggest current LLMs capture local formal patterns but struggle with global hierarchical planning required for robust linguistic‑aesthetic reasoning.","keyPoints":["Neo-Classic benchmark uses out‑of‑sample contemporary metrical poetry to test linguistic‑aesthetic reasoning.","State‑of‑the‑art LLMs drop 20‑50% performance on new texts and score 0‑13% on discourse ordering.","Expert guidance lifts reasoning‑enhanced models to 36% but still trails human experts."],"whyItMatters":"Identifying LLM weaknesses in higher‑order aesthetic reasoning guides future model improvements and informs reliable AI use in cultural, literary, and educational applications.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Qwen3-Max","Gemini-3-Pro","DeepSeek-V3.2"],"people":[]},"firstPublishedAt":"2026-09-18T04:00:00Z","updatedAt":"2026-09-18T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry","url":"https://arxiv.org/abs/2609.19154","publishedAt":"2026-09-18T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Neo-Classic benchmark evaluates linguistic-aesthetic reasoning in Classical Chinese poetry\", 18 September 2026, https://digestai.news/story/neo-classic-benchmark-evaluates-linguistic-aesthetic-reasoning-in-clas","publisher":"Digest AI","title":"Neo-Classic benchmark evaluates linguistic-aesthetic reasoning in Classical Chinese poetry","datePublished":"2026-09-18T04:00:00Z","url":"https://digestai.news/story/neo-classic-benchmark-evaluates-linguistic-aesthetic-reasoning-in-clas"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}