AWS shows how to cut agent costs with knowledge distillation on SageMaker

The process involves encoding trajectories as chat-formatted messages, running serverless supervised fine-tuning on SageMaker AI model customization, and evaluating the student against a held-out benchmark (ShoppingBench). Results show the 4B student matched the teacher’s Absolute Success Rate (ASR) of 0.541 on 324 held-out problems, with a slight edge over the sampled teacher’s 0.511. The student also preserved general instruction-following ability, though with a small drop (e.g., 0.741 vs. 0.819 on IFEval’s prompt-strict score). The guide emphasizes honest evaluation—comparing against both the teacher and independent frontier models—and highlights tradeoffs like task capacity limits and the need for on-policy methods for harder problems.