Live Stats, next update: Wed 02 Sep
Human PDBs Analysed
Confidently Wrong
Novel + Confidently Wrong
DB size
Visitors
Full statistics →
New PDB Depositions vs. Their Blind AlphaFold Predictions — A Running Test of “Is Folding Solved?”

Release week 2020-12-30

54
structures analysed (13 full · 24.1%)
00.0%
confidently wrong
00.0%
novel sequences
00.0%
novel & wrong
0.97
median TM-score

Zoomed into the red "confidently wrong" box above (pLDDT ≥ 70, TM < 0.5) — the structures AlphaFold got confidently wrong.

How to read this

Every point is one experimental protein structure. The horizontal axis is AlphaFold's own confidence in its prediction (mean pLDDT, 0–100). The vertical axis is how well that blind prediction actually matches the experiment (TM-score, 0–1; above 0.5 means the same fold, above 0.9 near-identical). Marker size grows with the FRAUD score (confidence-weighted error).

Cutoff: the shaded red box is the "confidently wrong" zone — AlphaFold was confident (mean pLDDT above 70) yet the fold is wrong (TM-score below 0.5). A predictor that had truly solved folding would leave that box empty.

Take-home: 0 of 54 structures (0.0%) are confidently wrong; median TM-score is 0.97.

Read moreShow less — how each metric is calculated

TM-score: how similar the two 3D shapes are overall, 0–1 (a random pair scores ~0.17, an identical fold ~1). Length-normalised so large and small proteins compare fairly. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) — dᵢ is the gap between the i-th aligned Cα atoms after best-fit superposition, and d₀ = 1.24(L−15)^⅓ − 1.8 sets the distance scale for length L. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264; computed with TM-align, doi:10.1093/nar/gki524.

pLDDT: AlphaFold's own confidence in each residue, 0–100 (higher = surer). It is the model predicting its own accuracy before ever seeing the experiment. We plot the per-structure mean. pLDDT = (1/N) Σᵢ pLDDTᵢ, where pLDDTᵢ is the network's confidence output for residue i.

FRAUD score: the headline number — how wrong the prediction was, weighted by how confident AlphaFold was, so a big error it was sure about counts most. FRAUD = (1/N) Σᵢ (pLDDTᵢ/100) · min(Δᵢ,15)/15, where Δᵢ is residue i's Cα distance from the experiment (Å) after superposition, capped at 15 Å. Runs 0 (perfect) to 1.

Novelty: how little AlphaFold had to go on — 100 minus the highest sequence identity between this protein and any structure released before the 2018-04-30 training cutoff. High = genuinely unseen (100% = nothing similar was ever in the training set). novelty = 100 − maxₚ identity(s, p) over pre-cutoff PDB chains p, with identity = 100 × (matching aligned residues)/(alignment length), from an MMseqs2 search.

Homology: the point colour. High novelty (above 70%) flags the sequence as novel (amber) — AlphaFold had no close template; otherwise a pre-cutoff homolog existed (blue) it could have learned the fold from. novel ⇔ novelty > 70%.

What the metrics mean

TM-score (0–1): overall fold match — above 0.5 is the same fold, above 0.9 near-identical. Cα-RMSD (Å): average backbone distance after best-fit superposition — lower is better (under 2 Å is excellent). lDDT (0–1): local accuracy measured without superposition — above 0.8 is good. Each bar counts how many structures fall in that range.

Take-home: median TM-score 0.97 — most predictions match the experimental fold well, with a long tail that do not.

Read moreShow less — how each metric is calculated

TM-score: overall shape match, 0–1 (above 0.5 = same fold, above 0.9 near-identical), length-normalised. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) with dᵢ the aligned-Cα gap after superposition and d₀ = 1.24(L−15)^⅓ − 1.8. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264.

Cα-RMSD: the average straight-line distance between matching backbone Cα atoms once the two structures are best-fit superposed (Å; lower is better, under ~2 Å excellent). RMSD = √( (1/N) Σᵢ |Pᵢ − (R·Qᵢ + t)|² ), with Pᵢ/Qᵢ the experimental/model Cα coordinates and R,t the rotation and translation from Kabsch superposition.

lDDT: local accuracy with no superposition — the fraction of short-range inter-residue distances the model reproduces, so it is not fooled by a single wrongly-placed domain. lDDT = (1/N) Σᵢ ¼ Σ_t 1[ |d_exp − d_model| < t ] over thresholds t ∈ {0.5, 1, 2, 4} Å and all residue pairs within 15 Å.

Trend over time

The blue line is the mean TM-score of the structures by their deposit month; the red bars count how many were confidently wrong. Binning by deposit date (rather than by when we processed them) gives an even timeline, tracking whether AlphaFold's accuracy on newly-deposited structures is holding steady, improving, or slipping as the PDB keeps growing. Months with fewer than 5 structures are omitted so each point is a meaningful average. A dashed trend line is drawn only if the change over time is statistically significant.

Read moreShow less — how each metric is calculated

Mean TM-score: the month's average shape-match score. TM̄ = (1/M) Σ TM over the M structures deposited that month.

Trend line: a monotonic-trend test over the monthly means — Mann-Kendall (Kendall's τ) for significance, with a robust Theil-Sen slope for the line. Rank-based, so it tolerates the skewed TM distribution and noisy low-count months. Shown only when p < 0.05; otherwise a note reports that no significant trend was found.

Confidently wrong: the count that were both confident and wrong. count( mean pLDDT > 70 AND TM < 0.5 ).

Matched structures

PDBUniProtProteinMethodÅDeposited Novelty %pLDDTTMlDDTGDT_TSRMSDFRAUDFlag
7BUL_A P32780 General transcription factor IIH subunit 1 NMR 2020-04-07 0.00 76.30 0.62 0.74 14.72 10.76 0.43 ok
6LZP_A Q12824 SWI/SNF-related matrix-associated actin-de NMR 2020-02-19 0.00 91.57 0.66 0.72 23.86 7.52 0.40 ok
7ASV_A Q9NVU0 DNA-directed RNA polymerase III subunit RP X-ray 1.55 2020-10-28 78.88 0.56 0.35 ok
6ZR7_AAA O60469 Down syndrome cell adhesion molecule X-ray 1.85 2020-07-11 69.56 0.72 0.20 ok
6Z7Y_F P01308 Insulin X-ray 2.20 2020-06-02 0.00 48.49 0.34 0.57 29.31 6.22 0.18 ok
6VTU_L Q6IPQ0 DH717.1 light chain X-ray 2.61 2020-02-13 89.38 0.83 0.15 ok
6Z7Y_E P01308 Insulin X-ray 2.20 2020-06-02 0.00 51.25 0.25 0.55 44.05 4.88 0.14 ok
6Z7W_J P01308 Insulin X-ray 2.42 2020-06-02 0.00 49.68 0.48 0.52 40.62 4.74 0.13 ok
6ZPH_B Q9NS87 Kinesin-like protein KIF15 EM 6.90 2020-07-08 72.12 0.82 0.13 ok
6VTU_H S6B291 DH717.1 heavy chain X-ray 2.61 2020-02-13 86.88 0.86 0.12 ok
6YW3_S Q16665 Hypoxia-inducible factor 1-alpha X-ray 2.28 2020-04-29 44.74 0.34 0.71 47.06 4.43 0.11 ok
6ZPH_A Q96EK5 KIF-binding protein EM 6.90 2020-07-08 90.31 0.88 0.10 ok
6Z7W_I P01308 Insulin X-ray 2.42 2020-06-02 0.00 51.59 0.29 0.71 57.81 2.70 0.09 ok
6ZPG_A Q96EK5 KIF-binding protein EM 4.60 2020-07-08 90.31 0.90 0.09 ok
6YE3_C P60568 Interleukin-2 X-ray 2.89 2020-03-23 84.12 0.92 0.06 ok
6YVX_A Q9GZT9 Egl nine homolog 1 X-ray 1.80 2020-04-28 71.88 0.91 0.06 ok
6YVW_A Q9GZT9 Egl nine homolog 1 X-ray 1.97 2020-04-28 71.88 0.92 0.06 ok
6YW0_A Q9GZT9 Egl nine homolog 1 X-ray 2.20 2020-04-29 71.88 0.92 0.06 ok
6YVZ_A Q9GZT9 Egl nine homolog 1 X-ray 1.91 2020-04-29 71.88 0.92 0.06 ok
5R69_A O95619 YEATS domain-containing protein 4 X-ray 1.83 2020-02-29 0.00 95.22 0.95 0.93 91.48 1.47 0.06 ok
7A8P_A O00411 DNA-directed RNA polymerase, mitochondrial EM 3.50 2020-08-30 83.44 0.94 0.05 ok
6Z6Y_A O00214 Galectin-8 X-ray 1.34 2020-05-29 90.69 0.95 0.04 ok
6ZPI_C Q9NS87 Kinesin-like protein KIF15 EM 4.50 2020-07-08 72.12 0.94 0.04 ok
7D8Z_A Q9H2X9 potassium-chloride cotransporter 2 EM 3.40 2020-10-11 78.44 0.95 0.04 ok
7D99_A Q9Y666 potassium-chloride co-transporter KCC4 EM 2.90 2020-10-12 80.81 0.96 0.03 ok
5R68_A O95619 YEATS domain-containing protein 4 X-ray 1.64 2020-02-29 0.00 95.67 0.98 0.97 97.69 0.66 0.03 ok
7A17_A O43426 Isoform 2 of Synaptojanin-1 X-ray 2.73 2020-08-12 67.06 0.96 0.03 ok
7A0V_A O43426 Synaptojanin-1 X-ray 2.30 2020-08-11 67.06 0.96 0.03 ok
7ASU_A Q9NVU0 DNA-directed RNA polymerase III subunit RP X-ray 2.23 2020-10-28 78.88 0.97 0.02 ok
6M07_A Q13093 Platelet-activating factor acetylhydrolase X-ray 2.64 2020-02-20 0.00 96.87 1.00 0.98 99.12 0.54 0.02 ok
7A17_C O43426 Isoform 2 of Synaptojanin-1 X-ray 2.73 2020-08-12 67.06 0.97 0.02 ok
6YW3_A Q9GZT9 Egl nine homolog 1 X-ray 2.28 2020-04-29 71.88 0.97 0.02 ok
6VAJ_A Q13526 Peptidyl-prolyl cis-trans isomerase NIMA-i X-ray 1.42 2019-12-17 91.62 0.98 0.02 ok
7JIS_A O00329 Phosphatidylinositol 4,5-bisphosphate 3-ki X-ray 2.42 2020-07-23 87.94 0.98 0.02 ok
7D90_A Q9UHW9 potassium-chloride cotransporter 3 EM 3.60 2020-10-12 79.94 0.98 0.02 ok
6M08_A Q13093 Platelet-activating factor acetylhydrolase X-ray 2.19 2020-02-20 0.00 96.85 1.00 0.99 99.73 0.35 0.02 ok
6M06_A Q13093 Platelet-activating factor acetylhydrolase X-ray 2.10 2020-02-20 0.00 96.82 1.00 0.99 99.39 0.51 0.02 ok
6M61_O P04406 Glyceraldehyde-3-phosphate dehydrogenase X-ray 1.82 2020-03-12 0.00 98.12 0.99 0.99 99.55 0.41 0.02 ok
6VBX_A Q07820 Induced myeloid leukemia cell differentiat X-ray 1.95 2019-12-19 63.62 0.98 0.02 ok
7B7N_E P29317 Ephrin type-A receptor 2 X-ray 2.69 2020-12-11 82.25 0.98 0.01 ok
6YW4_A Q9GZT9 Egl nine homolog 1 X-ray 1.53 2020-04-29 71.88 0.98 0.01 ok
6YW2_A Q9GZT9 Egl nine homolog 1 X-ray 2.14 2020-04-29 71.88 0.98 0.01 ok
6YW1_A Q9GZT9 Egl nine homolog 1 X-ray 1.46 2020-04-29 71.88 0.98 0.01 ok
6ZJZ_A Q9NR97 Toll-like receptor 8 X-ray 2.49 2020-06-29 86.12 0.99 0.01 ok
7KQQ_A Q9H6Y2 WD repeat-containing protein 55 X-ray 1.80 2020-11-17 87.06 0.99 0.01 ok
7KBH_A Q92769 Histone deacetylase 2 X-ray 2.68 2020-10-02 85.56 1.00 0.00 ok
6VVU_A Q15661 Tryptase alpha/beta-1 X-ray 3.00 2020-02-18 91.31 1.00 0.00 ok
6Z6A_A Q14145 Kelch-like ECH-associated protein 1 X-ray 2.37 2020-05-28 90.06 1.00 0.00 ok
7JWU_A P00352 Retinal dehydrogenase 1 X-ray 1.90 2020-08-26 97.81 1.00 0.00 ok
7KBG_A Q92769 Histone deacetylase 2 X-ray 1.26 2020-10-02 85.56 1.00 0.00 ok
7JWT_A P00352 Retinal dehydrogenase 1 X-ray 1.80 2020-08-26 97.81 1.00 0.00 ok
7JWS_A P00352 Retinal dehydrogenase 1 X-ray 1.60 2020-08-26 97.81 1.00 0.00 ok
7JWW_A P00352 Retinal dehydrogenase 1 X-ray 1.60 2020-08-26 97.81 1.00 0.00 ok
7JWV_A P00352 Retinal dehydrogenase 1 X-ray 1.60 2020-08-26 97.81 1.00 0.00 ok

Click a column header to sort. FRAUD = confidence-weighted error; a structure is flagged confidently wrong when mean pLDDT > 70 yet TM-score < 0.5.