# Amazon fine‑tunes GPT‑OSS‑20B on SageMaker HyperPod to reach 100% multilingual reasoning

Digest AI · Generative AI & Models · published 2026-10-20T17:48:27Z

Canonical: https://digestai.news/story/amazon-finetunes-gptoss20b-on-sagemaker-hyperpod-to-reach-100-multilin

## Summary

The base GPT‑OSS‑20B model, trained only in English, scored 22% correct reasoning-language on a 50‑example multilingual test (English 100%, French 0%, German 10%, Spanish 0%, Italian 0%). Using supervised fine‑tuning (SFT) on an Amazon SageMaker HyperPod cluster (4 × ml.g6e.12xlarge with L40S GPUs) and LoRA (r=8, α=16), the team raised reasoning-language correctness to 100% across all five languages. Answer‑language accuracy improved to 62% (31/50) after SFT. 

Applying Group Relative Policy Optimization (GRPO) on top of the SFT checkpoint added a further 8‑point gain, lifting answer‑language correctness to 70% while keeping reasoning at 100%. Per‑language answer accuracy after GRPO was French 70%, German 80%, Spanish 60%, Italian 40%, English 100%. A parallel experiment with Qwen2.5‑7B‑Instruct showed GRPO adds no benefit on a task (GSM8K) where the base model already scores 90.5%, confirming that GRPO requires headroom to be effective. The post also details the training stack, prerequisites, and cleanup steps for reproducibility.

## Key points

- Base GPT‑OSS‑20B achieved only 22% correct reasoning-language on multilingual test, defaulting to English.
- SFT on SageMaker HyperPod raised reasoning-language to 100% and answer-language to 62%.
- GRPO added an 8‑point boost, reaching 70% answer-language accuracy while preserving 100% reasoning.

## Why it matters

Shows that supervised fine‑tuning can eliminate language bias in open‑weight models, and that GRPO can modestly improve performance when the base model already shows some capability.

## Sources

1. [Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod](https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod) (AWS Machine Learning Blog, 2026-10-20, primary source)

## Cite

Digest AI, "Amazon fine‑tunes GPT‑OSS‑20B on SageMaker HyperPod to reach 100% multilingual reasoning", 20 October 2026, https://digestai.news/story/amazon-finetunes-gptoss20b-on-sagemaker-hyperpod-to-reach-100-multilin

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/amazon-finetunes-gptoss20b-on-sagemaker-hyperpod-to-reach-100-multilin.json