FoldCompare · CASPTracker · a weekly mode
BlindTest
Every week, one protein structure measured by experiment and released in the PDB after the AlphaFold Database had already published its prediction. Look only at the prediction's own confidence. Mark the stretches where you expect it to be wrong. Then the experiment is laid on the prediction and your marks are scored.
the Sketcher's drawing Prediction Computed by a model, with how sure it is. Not an experiment.
the Photographer's picture Experiment Measured from real molecules.
This week's issue
Loading this week's issue…
Back issues (all still open):
1. Read the prediction's confidence
The chain is cut into windows of 10 residues (UniProt numbering). Under each window are its residues' pLDDT bands: the model's own score for how sure it is of each residue's local shape.
Optional: the PAE map, window by window
PAE (predicted aligned error) is the model's own guess, in Å, of how far off one part would be if you lined up on another. Darker means a smaller expected error. Two dark blocks with a pale band between them often mean two parts that are each sure of their shape but not of how they sit together.
2. Mark the windows you expect to be off
"Off" means: after the experiment and the prediction are laid on each other, the window's CA atoms (one per residue) sit more than 3 Å from the experiment on average. Tap a window to mark it; tap again to unmark. Marking none is a fair answer. Grey windows have fewer than 5 residues in the experiment and are not scored.
How an issue is chosen
- An experimental structure released in the PDB on or after 2025-06-01: X-ray crystallography at 2.5 Å or better, or cryo-electron microscopy at 3.2 Å or better; one kind of protein chain, 80 to 450 residues, matched to one reviewed (Swiss-Prot) UniProt entry.
- The blind filter used for these issues: This PDB entry is the first experimental structure of the protein in the PDB, and the AlphaFold DB model is dated before the entry was released.
- One issue per release month: the first entry of that month that passes. Issues are numbered oldest first, so an issue's number never changes.
- The experiment's residues are paired with the model's by UniProt number (the PDB's SIFTS mapping), and their letters agree on at least 95 of every 100 pairs.
- The two are laid on each other by a least-squares fit (the Kabsch method) on the residues the model scored 70 or more, then every paired residue's CA distance is measured. The same code makes the answers and checks them at build time.
When the targets and results of CASP17 are public, they will be added here as an issue.
Honest notes
- "Blind" here means no structure of this exact UniProt entry was in the PDB first. A close relative may have had one, and AlphaFold can draw on relatives. So this is gentler than CASP, where targets are kept secret.
- The AlphaFold DB model is one chain on its own. An experiment may hold several chains, a partner, a membrane or a bound molecule, so a window can be off for reasons that are not the model's fault.
- When two parts sit at a different angle, every window of one part can read "off" even if each part's own shape matches. The PAE map is where that shows.
- The experiment is a measurement with its own limits: a crystal packs copies against each other, and a few residues may be missing.
- Scores are counts beside what marking at random would score. One issue is a handful of windows: too few to say how good anyone is at this.
Sources
- Experimental structures and SIFTS mapping: RCSB Protein Data Bank (Burley et al. 2023, Nucleic Acids Res.).
- Predictions, pLDDT and PAE: AlphaFold Protein Structure Database (Varadi et al. 2024, Nucleic Acids Res.; CC BY 4.0).
- What pLDDT and PAE mean: Jumper et al. 2021, Nature 596:583.
- Protein names and organisms: UniProt.
- The superposition: Kabsch 1976, Acta Cryst. A32:922.
- The blind experiment this borrows from: CASP, the Protein Structure Prediction Center.
Snapshot 2026-10-08 · 12 issues.