Live Stats, next update: Wed 02 Sep
Human PDBs Analysed
Confidently Wrong
Novel + Confidently Wrong
DB size
Visitors
Full statistics →
New PDB Depositions vs. Their Blind AlphaFold Predictions — A Running Test of “Is Folding Solved?”

Release week 2018-11-28

88
structures analysed (61 full · 69.3%)
78.0%
confidently wrong
00.0%
novel sequences
00.0%
novel & wrong
0.928
median TM-score

Zoomed into the red "confidently wrong" box above (pLDDT ≥ 70, TM < 0.5) — the structures AlphaFold got confidently wrong.

The worst offenders ranked, one row per protein (AlphaFold has one model per sequence, so repeat depositions are collapsed — “×N” marks how many structures of that protein exist; the worst is shown). Each row joins what AlphaFold claimed (blue, pLDDT/100) to what the experiment showed (red, TM-score) — the longer the bar, the bigger the miss. Hover a dot to preview its Cα-deviation ribbon; click to open the full entry.

How to read this

Every point is one experimental protein structure. The horizontal axis is AlphaFold's own confidence in its prediction (mean pLDDT, 0–100). The vertical axis is how well that blind prediction actually matches the experiment (TM-score, 0–1; above 0.5 means the same fold, above 0.9 near-identical). Marker size grows with the FRAUD score (confidence-weighted error).

Cutoff: the shaded red box is the "confidently wrong" zone — AlphaFold was confident (mean pLDDT above 70) yet the fold is wrong (TM-score below 0.5). A predictor that had truly solved folding would leave that box empty.

Take-home: 7 of 88 structures (8.0%) are confidently wrong; median TM-score is 0.928.

Read moreShow less — how each metric is calculated

TM-score: how similar the two 3D shapes are overall, 0–1 (a random pair scores ~0.17, an identical fold ~1). Length-normalised so large and small proteins compare fairly. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) — dᵢ is the gap between the i-th aligned Cα atoms after best-fit superposition, and d₀ = 1.24(L−15)^⅓ − 1.8 sets the distance scale for length L. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264; computed with TM-align, doi:10.1093/nar/gki524.

pLDDT: AlphaFold's own confidence in each residue, 0–100 (higher = surer). It is the model predicting its own accuracy before ever seeing the experiment. We plot the per-structure mean. pLDDT = (1/N) Σᵢ pLDDTᵢ, where pLDDTᵢ is the network's confidence output for residue i.

FRAUD score: the headline number — how wrong the prediction was, weighted by how confident AlphaFold was, so a big error it was sure about counts most. FRAUD = (1/N) Σᵢ (pLDDTᵢ/100) · min(Δᵢ,15)/15, where Δᵢ is residue i's Cα distance from the experiment (Å) after superposition, capped at 15 Å. Runs 0 (perfect) to 1.

Novelty: how little AlphaFold had to go on — 100 minus the highest sequence identity between this protein and any structure released before the 2018-04-30 training cutoff. High = genuinely unseen (100% = nothing similar was ever in the training set). novelty = 100 − maxₚ identity(s, p) over pre-cutoff PDB chains p, with identity = 100 × (matching aligned residues)/(alignment length), from an MMseqs2 search.

Homology: the point colour. High novelty (above 70%) flags the sequence as novel (amber) — AlphaFold had no close template; otherwise a pre-cutoff homolog existed (blue) it could have learned the fold from. novel ⇔ novelty > 70%.

What the metrics mean

TM-score (0–1): overall fold match — above 0.5 is the same fold, above 0.9 near-identical. Cα-RMSD (Å): average backbone distance after best-fit superposition — lower is better (under 2 Å is excellent). lDDT (0–1): local accuracy measured without superposition — above 0.8 is good. Each bar counts how many structures fall in that range.

Take-home: median TM-score 0.928 — most predictions match the experimental fold well, with a long tail that do not.

Read moreShow less — how each metric is calculated

TM-score: overall shape match, 0–1 (above 0.5 = same fold, above 0.9 near-identical), length-normalised. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) with dᵢ the aligned-Cα gap after superposition and d₀ = 1.24(L−15)^⅓ − 1.8. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264.

Cα-RMSD: the average straight-line distance between matching backbone Cα atoms once the two structures are best-fit superposed (Å; lower is better, under ~2 Å excellent). RMSD = √( (1/N) Σᵢ |Pᵢ − (R·Qᵢ + t)|² ), with Pᵢ/Qᵢ the experimental/model Cα coordinates and R,t the rotation and translation from Kabsch superposition.

lDDT: local accuracy with no superposition — the fraction of short-range inter-residue distances the model reproduces, so it is not fooled by a single wrongly-placed domain. lDDT = (1/N) Σᵢ ¼ Σ_t 1[ |d_exp − d_model| < t ] over thresholds t ∈ {0.5, 1, 2, 4} Å and all residue pairs within 15 Å.

Trend over time

The blue line is the mean TM-score of the structures by their deposit month; the red bars count how many were confidently wrong. Binning by deposit date (rather than by when we processed them) gives an even timeline, tracking whether AlphaFold's accuracy on newly-deposited structures is holding steady, improving, or slipping as the PDB keeps growing. Months with fewer than 5 structures are omitted so each point is a meaningful average. A dashed trend line is drawn only if the change over time is statistically significant.

Read moreShow less — how each metric is calculated

Mean TM-score: the month's average shape-match score. TM̄ = (1/M) Σ TM over the M structures deposited that month.

Trend line: a monotonic-trend test over the monthly means — Mann-Kendall (Kendall's τ) for significance, with a robust Theil-Sen slope for the line. Rank-based, so it tolerates the skewed TM distribution and noisy low-count months. Shown only when p < 0.05; otherwise a note reports that no significant trend was found.

Confidently wrong: the count that were both confident and wrong. count( mean pLDDT > 70 AND TM < 0.5 ).

Matched structures

PDBUniProtProteinMethodÅDeposited Novelty %pLDDTTMlDDTGDT_TSRMSDFRAUDFlag
6MZM_K Q7Z7C8 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 16.50 88.18 0.47 0.75 1.49 27.04 0.83 wrong
6MZL_K Q7Z7C8 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 16.50 88.18 0.47 0.75 1.49 26.32 0.82 wrong
6MZC_K Q7Z7C8 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 16.50 88.18 0.47 0.75 1.49 26.32 0.82 wrong
6MZL_A P21675 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 1.20 86.86 0.85 0.92 3.69 29.36 0.76 ok
6MZL_H P49848 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 82.27 0.75 0.83 3.57 20.81 0.71 ok
6N32_H P0DOX5 Fab 2G12 heavy chain X-ray 2.20 2018-11-14 0.00 91.75 0.53 0.91 5.40 13.12 0.68 ok
6N2X_H P0DOX5 Fab 2G12 heavy chain X-ray 3.00 2018-11-14 0.00 91.69 0.53 0.90 5.75 13.04 0.68 ok
6MZM_W P52655 Transcription initiation factor IIA subuni EM 7.50 2018-11-05 10.00 77.27 0.94 0.54 2.22 16.04 0.66 ok
6MZL_I P49848 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 1.50 81.83 0.73 0.81 16.45 15.31 0.47 ok
6MZC_I P49848 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 1.50 81.83 0.73 0.81 16.45 15.31 0.47 ok
6MZM_I P49848 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 1.50 81.83 0.73 0.81 17.48 14.91 0.46 ok
6MZM_D O00268 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 88.74 0.54 0.89 28.71 6.38 0.34 ok
6MZL_F Q15542 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 90.80 0.88 0.64 39.32 9.28 0.31 ok
6MZD_F Q15542 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 90.80 0.88 0.64 39.32 9.28 0.31 ok
6MZC_G Q15542 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 0.00 90.80 0.88 0.65 39.27 9.27 0.31 ok
6MZM_G Q15542 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 90.80 0.88 0.65 39.75 9.29 0.31 ok
6N13_B O60260 E3 ubiquitin-protein ligase parkin NMR 2018-11-08 0.00 85.80 0.77 0.82 36.65 5.06 0.26 ok
6MZL_D O00268 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 89.70 0.62 0.90 46.32 4.05 0.21 ok
6MZD_D O00268 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 89.70 0.62 0.90 46.32 4.05 0.21 ok
6MZC_E O00268 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 0.00 89.70 0.63 0.90 48.28 4.02 0.21 ok
6MZL_L Q16594 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 36.80 91.25 0.86 0.86 69.95 2.69 0.12 ok
6MZD_L Q16594 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 36.80 91.25 0.86 0.86 69.95 2.69 0.12 ok
6MZC_M Q16594 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 36.80 91.25 0.86 0.86 70.43 2.68 0.12 ok
6MZM_L Q16594 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 36.80 91.25 0.86 0.86 70.43 2.67 0.12 ok
6MZM_R Q16514 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 4.00 94.53 0.83 0.80 75.00 1.92 0.10 ok
6MZL_C Q5VWG9 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 3.00 92.09 0.83 0.79 76.08 2.06 0.10 ok
6MZD_C Q5VWG9 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 3.00 92.09 0.83 0.79 76.08 2.06 0.10 ok
6MZC_R Q16514 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 4.00 94.53 0.83 0.80 75.34 1.89 0.10 ok
6MZL_Q Q16514 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 4.00 94.53 0.82 0.81 76.35 1.86 0.10 ok
6MZD_Q Q16514 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 4.00 94.53 0.82 0.81 76.35 1.86 0.10 ok
6MZM_B Q6P1X5 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 83.12 0.97 0.75 78.77 2.23 0.08 ok
6MZC_O Q12962 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 0.00 92.15 0.87 0.83 82.32 1.82 0.08 ok
6MZM_O Q12962 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 92.15 0.87 0.83 81.71 1.82 0.08 ok
6MZL_B Q6P1X5 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 83.12 0.97 0.75 79.24 2.20 0.08 ok
6MZC_B Q6P1X5 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 0.00 83.12 0.97 0.75 79.24 2.20 0.08 ok
6MZL_N Q12962 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 92.15 0.87 0.83 82.01 1.82 0.08 ok
6MZD_N Q12962 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 92.15 0.87 0.83 82.01 1.82 0.08 ok
6IPY_A P06241 Tyrosine-protein kinase Fyn X-ray 1.34 2018-11-05 1.60 90.13 0.89 0.89 87.31 2.53 0.08 ok
6MZD_A P21675 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 60.73 0.77 0.81 74.30 2.10 0.07 ok
6MZM_H P49848 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 81.44 0.95 0.82 84.34 1.62 0.06 ok
6MZC_H P49848 Transcription initiation factor TFIID subu EM 4.50 2018-11-05 0.00 81.44 0.95 0.82 84.53 1.58 0.06 ok
6MZL_P Q15544 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 94.88 0.92 0.92 89.04 1.08 0.06 ok
6MZD_P Q15544 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 94.88 0.92 0.92 89.04 1.08 0.06 ok
6MZD_H P49848 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 85.23 0.89 0.87 88.54 1.22 0.06 ok
6N13_A P0CG48 phosphoubiquitin NMR 2018-11-08 0.00 89.83 0.93 0.89 91.78 1.71 0.06 ok
6E7Z_A Q9GZU1 Mucolipin-1 EM 3.73 2018-07-27 81.25 0.93 0.06 ok
6N13_D P0CG47 ubiquitin NMR 2018-11-08 0.00 94.12 0.93 0.89 91.78 1.12 0.05 ok
6MZM_J Q15545 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 91.53 0.96 0.94 93.29 0.92 0.04 ok
6MZL_J Q15545 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 91.53 0.96 0.94 93.29 0.92 0.04 ok
6N13_C P68036 Ubiquitin-conjugating enzyme E2 L3 NMR 2018-11-08 0.70 95.54 0.98 0.90 96.59 0.74 0.04 ok
6MZM_X P52657 Transcription initiation factor IIA subuni EM 7.50 2018-11-05 0.00 97.47 0.97 0.98 97.42 0.70 0.04 ok
6MZL_S Q15543 Transcription initiation factor TFIID subu EM 23.00 2018-11-05 0.00 96.38 0.93 0.97 100.00 0.59 0.04 ok
6MZD_S Q15543 Transcription initiation factor TFIID subu EM 9.80 2018-11-05 0.00 96.38 0.93 0.97 100.00 0.59 0.04 ok
6IPZ_Z P06241 Tyrosine-protein kinase Fyn X-ray 1.58 2018-11-05 1.60 89.14 0.93 0.94 94.05 1.09 0.03 ok
6MZM_A P21675 Transcription initiation factor TFIID subu EM 7.50 2018-11-05 0.00 91.58 0.99 0.98 97.19 0.87 0.03 ok
6E7Y_A Q9GZU1 Mucolipin-1 EM 3.57 2018-07-27 81.25 0.96 0.03 ok
6MZL_T P20226 TATA-box-binding protein EM 23.00 2018-11-05 0.00 95.61 0.99 0.96 98.47 0.59 0.03 ok
6MZD_T P20226 TATA-box-binding protein EM 9.80 2018-11-05 0.00 95.61 0.99 0.96 98.47 0.59 0.03 ok
6E7P_A Q9GZU1 Mucolipin-1 EM 3.50 2018-07-27 81.25 0.96 0.03 ok
6IGW_A O95297 Myelin protein zero-like protein 1 X-ray 1.98 2018-09-26 76.38 0.96 0.03 ok
6MZM_T P20226 TATA-box-binding protein EM 7.50 2018-11-05 0.00 95.61 0.99 0.98 99.31 0.52 0.03 ok
6N35_H P01857 Fab 2G12 heavy chain X-ray 1.75 2018-11-14 0.00 90.77 0.44 0.95 97.37 0.88 0.03 wrong
6N35_K P01834 Fab 2G12 light chain X-ray 1.75 2018-11-14 0.00 97.24 0.49 0.97 98.35 0.57 0.03 wrong
6DB4_A P52333 Tyrosine-protein kinase JAK3 X-ray 1.66 2018-05-02 85.69 0.97 0.03 ok
6N32_L P01834 Fab 2G12 light chain X-ray 2.20 2018-11-14 0.00 97.24 0.50 0.97 98.58 0.53 0.03 wrong
6IGT_A O95297 Myelin protein zero-like protein 1 X-ray 2.40 2018-09-26 76.38 0.97 0.03 ok
6N2X_K P01834 Fab 2G12 light chain X-ray 3.00 2018-11-14 0.00 97.24 0.49 0.98 98.82 0.46 0.02 wrong
6DA4_A P52333 Tyrosine-protein kinase JAK3 X-ray 2.90 2018-05-01 85.69 0.97 0.02 ok
6DUD_A P52333 Tyrosine-protein kinase JAK3 X-ray 1.66 2018-06-20 85.69 0.98 0.02 ok
6HPW_A Q03111 Protein ENL X-ray 1.90 2018-09-22 64.69 0.97 0.02 ok
6HKR_A P29373 Cellular retinoic acid-binding protein 2 X-ray 1.50 2018-09-07 96.75 0.98 0.02 ok
6DB3_A P52333 Tyrosine-protein kinase JAK3 X-ray 1.97 2018-05-02 85.69 0.98 0.02 ok
6HPX_A Q03111 Protein ENL X-ray 2.30 2018-09-22 64.69 0.98 0.02 ok
6H1H_A O00625 Pirin X-ray 1.54 2018-07-11 96.94 0.99 0.01 ok
6HPZ_A Q03111 Protein ENL X-ray 2.30 2018-09-22 64.69 0.98 0.01 ok
6IGO_A O95297 Myelin protein zero-like protein 1 X-ray 2.75 2018-09-25 76.38 0.98 0.01 ok
6HPY_A Q03111 Protein ENL X-ray 2.00 2018-09-22 64.69 0.98 0.01 ok
6I2K_E Q14108 Lysosome membrane protein 2 EM 3.40 2018-11-01 92.75 0.99 0.01 ok
6H1I_A O00625 Pirin X-ray 1.69 2018-07-11 96.94 0.99 0.01 ok
6HQ0_A Q03111 Protein ENL X-ray 1.81 2018-09-22 64.69 0.99 0.01 ok
6EDA_A P00918 Carbonic anhydrase 2 X-ray 1.88 2018-08-09 97.38 1.00 0.00 ok
6EBE_A P00918 Carbonic anhydrase 2 X-ray 1.88 2018-08-06 97.38 1.00 0.00 ok
6EEA_A P00918 Carbonic anhydrase 2 X-ray 1.63 2018-08-13 97.38 1.00 0.00 ok
6ECZ_A P00918 Carbonic anhydrase 2 X-ray 2.21 2018-08-08 97.38 1.00 0.00 ok
6EEO_A P00918 Carbonic anhydrase 2 X-ray 1.72 2018-08-15 97.38 1.00 0.00 ok
6EEH_A P00918 Carbonic anhydrase 2 X-ray 1.63 2018-08-14 97.38 1.00 0.00 ok
6HH6_A Q86W56 Poly(ADP-ribose) glycohydrolase X-ray 1.85 2018-08-24 68.25 1.00 0.00 ok
6IOZ_A P22304 Iduronate 2-sulfatase X-ray 3.10 2018-11-01 93.06 1.00 0.00 ok

Click a column header to sort. FRAUD = confidence-weighted error; a structure is flagged confidently wrong when mean pLDDT > 70 yet TM-score < 0.5.