GPT-6 Astra edits neural network weights to improve performance without training
Researchers report that GPT-6 Astra can read and modify neural network weights directly, achieving performance gains in tasks like game-playing and math reasoning without traditional training methods. In experiments, Astra identified and fixed issues in small networks, such as a car-driving policy stuck in a local minimum, and generated weights for new networks that outperformed those trained…
Key points
- GPT-6 Astra edits neural network weights to fix failures like local minima in reinforcement learning policies, improving performance without retraining
- On Qwen3.5-0.8B, Astra corrected reasoning loops by modifying 0.016% of weights, raising MATH-500 from 7.9% to 48.1% with 50 forward passes
- Researchers trained Astra to generate weights for game-playing networks (Breakout, Seaquest) that outperformed PPO-trained models by 20x in some cases
The work, titled Sparks of Weight-Space Programming, demonstrates Astra’s ability to interpret weight structures—such as attention patterns in language models or geometric features in image-generating networks—without running forward passes. While limitations remain (e.g., context length constraints and reliance on forward passes for larger models), the authors argue this marks a shift toward LLMs as native weight-space optimizers. They plan to release benchmarks and harnesses to standardize evaluation, citing discussions with researchers at Berkeley, CMU, and cmpnd.
Model page: GPT-6 Astra →
The story so far
5 episodes →- GPT-6 Astra edits neural network weights to improve performance without trainingthis story
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
rikiyat.github.io · 2 November 2026
Loading the full article…
This text was published by rikiyat.github.io and written by Rikiya Takehi. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers demonstrate pretraining poisoning backdoors in Qwen3 agents · 1 src
- Researchers introduce PathLUPI to improve molecular prediction from histology images · 1 src
- MIT researchers win 2026 Pritzker Prize for AI antibiotic work · 1 src
- Empero Research releases PiVo-1.4M, a 1.4M-parameter text-to-speech model for local use · 1 src
- Unite.AI explains AI latency and throughput metrics for engineers · 1 src
Comments
via GitHub Discussions