{"version":1,"type":"story","url":"https://digestai.news/story/arxiv-study-tests-how-ai-models-improve-data-science-projects-with-fee","json":"https://digestai.news/story/arxiv-study-tests-how-ai-models-improve-data-science-projects-with-fee.json","markdown":"https://digestai.news/story/arxiv-study-tests-how-ai-models-improve-data-science-projects-with-fee.md","slug":"arxiv-study-tests-how-ai-models-improve-data-science-projects-with-fee","headline":"arXiv study tests how AI models improve data science projects with feedback","summary":"Researchers on arXiv examined whether agentic AI models can complete full data science projects and how feedback affects their performance. A ‘writer’ model generated projects—including data download, cleaning, analysis, and a short paper—while a ‘grader’ model evaluated the work. Each model served as both writer and grader, creating 49 pairs tested over three rounds of iteration.\n\nOf the 147 initial submissions, 107 ran successfully. Scores rose from an average of 59% to 76% after the final revision. Improvements varied: Anthropic’s Opus 5.5 saw a 3-point gain, while Haiku 4.5 (also from Anthropic) improved by 46 points. Grading by the same model as the writer boosted scores by about 9 percentage points. The study found feedback generally helped, but weaker models did not consistently improve stronger ones.","keyPoints":["7 models from 5 labs—Anthropic, OpenAI, Google, DeepSeek, Z.AI—tested as writers and graders in 49 pairs","107 of 147 initial submissions ran successfully, scoring 59% on average before feedback","Scores rose to 76% after three rounds, with Haiku 4.5 improving by 46 points"],"whyItMatters":"The study shows how iterative feedback and model self-correction can improve AI-generated data science work, with varying results depending on the model’s role and strength.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Anthropic","OpenAI","Google","DeepSeek","Z.AI"],"models":["Opus 5.5","Haiku 4.5","Gemini 3.8 Flash"],"people":[]},"firstPublishedAt":"2026-11-05T05:00:00Z","updatedAt":"2026-11-05T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"Iterated creation, grading, and revision of data science projects by language models","url":"https://arxiv.org/abs/2611.05548","publishedAt":"2026-11-05T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"arXiv study tests how AI models improve data science projects with feedback\", 5 November 2026, https://digestai.news/story/arxiv-study-tests-how-ai-models-improve-data-science-projects-with-fee","publisher":"Digest AI","title":"arXiv study tests how AI models improve data science projects with feedback","datePublished":"2026-11-05T05:00:00Z","url":"https://digestai.news/story/arxiv-study-tests-how-ai-models-improve-data-science-projects-with-fee"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}