Researchers release ProBEiT, a 400M-parameter protein model
Researchers have introduced ProBEiT, a multimodal protein language model designed to integrate structural information with amino-acid sequences. The architecture uses an implicit mixture-of-experts scheme with deterministic routing to process discrete, tokenized structures alongside sequence data. This approach aims to improve the prediction of protein function and mutational effects without the…
Key points
- ProBEiT is a 400 million-parameter multimodal model trained on 1.7 million AlphaFoldDB proteins.
- It uses an implicit mixture-of-experts scheme to jointly encode sequences and tokenized structures.
- The model outperforms larger baselines on six tasks and identifies optimized protein variants in experiments.
ProBEiT was trained on 1.7 million proteins from the AlphaFoldDB. With 400 million parameters, the model reportedly surpasses larger baselines across six tasks involving fitness prediction and functional annotation. The authors note that ProBEiT achieves strong accuracy early in the continual training period, which reduces overall computational requirements compared to billion-scale models.
In experimental tests, the model was applied to thermostability engineering of glycerol-3-phosphate oxidase. It successfully identified optimized variants with positive epistatic effects. The study, published in Nature Machine Learning, suggests that tightly coupling structural context with sequence modeling in a modality-specialized framework enables efficient protein engineering.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- iMerit launches Scholars platform to scale expert AI training and evaluation · 1 src
- Machine learning explores quantum superpositions but cannot solve Schrödinger’s cat paradox · 1 src
- KDnuggets outlines four-stage roadmap for computer Vision engineers · 1 src
- Resolution autoformalizes math to speed AI alignment research · 1 src
- Graphite finds ChatGPT still rarely mentions many brands even when using search · 1 src
Comments
via GitHub Discussions