ESMFold2 uses transformer embeddings to predict structures of proteins, DNA, RNA and ligands

The nonprofit biomedical research organization Biohub and the AI-for-biology lab EvolutionaryScale have released ESMFold2, an open-source system that predicts three-dimensional structures of biologically active molecules — including proteins, DNA, RNA and molecules that bind to them — by treating their components like elements of language. The core idea is that a transformer-produced embedding can substitute for a multiple sequence alignment (MSA), which is useful when related sequences are scarce or absent.

Capabilities

• Inputs: amino-acid sequences (proteins), base-pair sequences (DNA/RNA), SMILES descriptions of other bioactive molecules, or a multiple sequence alignment (MSA) of related sequences.
• Outputs: atom coordinates (molecular shape) and error estimates for those coordinates.
• Architecture: mixed-model with about 6.2 billion parameters.
• Availability: free to use via the project website, weights downloadable from HuggingFace, and an API provided via Biohub.

How it works — overview

ESMFold2 processes inputs in several stages:

1. It produces three types of embeddings: (a) a sequence embedding using the ESMC transformer, (b) an atom embedding using a separate transformer, and (c) an optional MSA embedding using a pairmixer module.
2. Another pairmixer uses these to produce a pairwise-distance embedding representing distances between amino acids, base pairs or atoms. This distance embedding is initialized as noise; during training it was refined by up to six cycles, while at inference the model cycles the embedding ten times because that gave the best performance.
3. A diffusion model uses the distance embedding, sequence and atom embeddings, and a noisy point cloud to denoise and infer atom positions.
4. A third pairmixer estimates errors, including mismatches between predicted and actual inter-atom distances.

Key components and training data

• ESMC: the transformer that embeds sequences. It was trained to fill in masked tokens using roughly 2.8 billion sequences drawn from three protein databases.
• A separate transformer embeds atoms.
• Pairmixer: a module that updates each element of a matrix based on elements in the same row or column; used for MSA embedding and for constructing pairwise distance embeddings.
• The model was trained on two datasets that map existing sequences and descriptions to known structures.

Performance (FoldBench evaluations)

• With only protein sequence input (no MSA), ESMFold2 outperformed Chai-1, a model that does not accept MSAs. On Local Distance Difference Test (lDDT), which measures similarity between predicted inter-atom distances and ground truth (higher is better), ESMFold2 scored 0.85 while Chai-1 scored 0.81.
• With an MSA available, ESMFold2 achieved 0.89 lDDT, matching the performance of AlphaFold3 and Protenix-v1.
• DockQ pass rate (similarity at molecular contact points): for a bound protein–DNA pair with no MSA, ESMFold2 achieved an 80% pass rate versus Chai-1’s 71%. With an MSA, ESMFold2 reached 79%, matching Protenix-v1 but trailing AlphaFold3’s 82%.

What differentiates ESMFold2 and why it matters

AlphaFold3 and similar models typically require an MSA, which depends on finding and aligning related sequences in databases. ESMFold2’s main innovation is using a transformer (ESMC) to embed individual molecules in a language-model style, allowing accurate structure prediction without an MSA. This lowers friction for biological research, especially for novel or rapidly evolving molecules (for example viral proteins) or synthetic molecules where related sequences may be rare or nonexistent.

Another practical advantage is open access: the model weights are public, enabling researchers across institutions and funding levels to use the system.

Background and summary

ESMFold2 is an evolution of Biohub’s 2022 ESMFold: a larger model trained on more data and incorporating high-performing components from other work (notably AlphaFold3), such as a diffusion-based atom coordinate predictor and an explicit error-estimation module. Evaluation on standard benchmarks shows ESMFold2 is especially competitive when MSAs are unavailable and remains on par with leading models when MSAs are provided.

The developers also note that applying multiple processing cycles at inference — a technique familiar from large language models — improves distance estimation; ESMFold2 applies this principle by cycling its distance embedding multiple times during inference.