SimpleDesign model unifies protein sequence and structure design in a single-stage transformer
SimpleDesign, introduced by Apple Machine Learning Research, is a transformer‑based multimodal model that generates protein sequences and their three‑dimensional structures together. Unlike prior pipelines that first compress data with autoencoders and then train a generative model on latent codes, SimpleDesign uses a single‑stage end‑to‑end loss that combines discrete cross‑entropy for…
Key points
- SimpleDesign trains directly on raw sequence‑structure pairs, eliminating the need for separate autoencoders
- Model is built on a transformer backbone with modality‑specific processing and shared global attention
- Trained on >2 million protein pairs, achieving state‑of‑the‑art results on co‑design benchmarks
The work demonstrates that the extra complexity of multi‑stage training is not required for high‑quality protein design, potentially lowering computational costs and simplifying future research. By keeping global self‑attention across both modalities, SimpleDesign can capture intricate dependencies between sequence motifs and structural folds, offering a new tool for drug discovery and synthetic biology.
SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
Apple Machine Learning Research · 11 September 2026
SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
AuthorsJiarui Lu†, Yuyang Wang, Yizhe Zhang, Jiatao Gu**, Navdeep Jaitly**, Joshua M. Susskind, Miguel Ángel Bautista
Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Developing generative models capable of understanding this intrinsically multi-modal relationship is crucial for fields like drug discovery and protein engineering. Existing models often rely on a multi-stage training process where autoencoders that tokenize data into latent representations are trained in a first stage. Secondly, a generative model is trained on the latent representation of the autoencoder(s), i.e. generative modeling in a latent space. We hypothesize that this multi-stage training is not necessary to obtain performant co-design models and thus present SimpleDesign, an effective multi-modal protein design model trained directly in the data space. SimpleDesign leverages a single-stage end-to-end objective that combines discrete cross-entropy for sequences and a regression objective for structures. In order to effectively model the difference in sequence and structure modalities, we instantiate the framework with Transformer-based multimodal backbones that allows modality-specific processing while keeping global self-attention over both modalities. We train SimpleDesign on over 2M sequence-structure pairs achieving competitive performance across co-design and unconditional sequence/structure generation benchmarks.
SimpleFold: Folding Proteins is Simpler than You Think
September 24, 2025research area Methods and Algorithmsconference ICLR, conference ICML
Protein folding models have achieved groundbreaking results since the introduction of AlphaFold2, typically built via a combination of integrating domain-expertise into its architectural designs and training pipelines. Nonetheless, given the success of generative models across different but related problems, it is natural to question whether these architectural designs are a necessity to build performant models. In this paper, we introduce…
INRFlow: Flow Matching for INRs in Ambient Space
June 27, 2025research area Methods and Algorithmsconference ICML
Flow matching models have emerged as a powerful method for generative modeling on domains like images or videos, and even on irregular or unstructured data like 3D point clouds or even protein structures. These models are commonly trained in two stages: first, a data compressor is trained, and in a subsequent training stage a flow matching generative model is trained in the latent space of the data compressor. This two-stage paradigm sets…
This text was published by Apple Machine Learning Research . It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
Comments
via GitHub Discussions