Study finds GPT-2-small learns action nominal constructions, mirroring typological patterns
Researchers built a grammar‑engineered synthetic corpus that blends shared semantic input from natural texts with computational grammars. Using this pipeline they generated 96 artificial languages that vary in clause word order, noun‑phrase order, alignment, complement system, and action nominal construction (ANC) strategy. For each language they trained a GPT‑2‑small model from scratch.
Key points
- Trained 96 GPT-2-small models from scratch on synthetic corpora covering varied clause and noun‑phrase orders.
- Evaluated with 34 targeted minimal‑pair tests, finding parallels in clause configurations but divergences in complement systems and ANC strategies.
- Models also learned feature combinations that are rare or unattested in natural languages.
The models were evaluated with targeted minimal‑pair tests covering 34 grammatical phenomena. Results show that the models reproduce typological parallels in basic clause configurations, but diverge in complement systems and ANC strategies. Moreover, the models sometimes learn feature combinations that are rare or unattested in real languages. The authors argue that these partial alignments suggest natural language distributions reflect both general learning pressures and functional demands that are not fully captured by current model‑training experiments.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic says small AI may self‑replicate by year‑end after experiment with GLM‑5.2 · 1 src
- Anthropic co-founder Jack Clark says AI could boost US GDP by nearly 50% by 2030 · 13 src
- Hybrid Reasoning Agents improve LLM performance by 10.4% and reduce costs up to 3.7x · 1 src
- Chinese team proves 3D Ghys conjecture using Intern‑S2 AI model · 2 src
- StarSkirmish Bench v0.2 ranks GPT-6 Astra as top StarCraft bot writer · 2 src
Comments
via GitHub Discussions