Live Stats, next update: Wed 02 Sep
Human PDBs Analysed
Confidently Wrong
Novel + Confidently Wrong
DB size
Visitors
Full statistics →
New PDB Depositions vs. Their Blind AlphaFold Predictions — A Running Test of “Is Folding Solved?”

Release week 2020-06-24

77
structures analysed (37 full · 48.1%)
45.2%
confidently wrong
33.9%
novel sequences
22.6%
novel & wrong
0.979
median TM-score

Zoomed into the red "confidently wrong" box above (pLDDT ≥ 70, TM < 0.5) — the structures AlphaFold got confidently wrong.

The worst offenders ranked, one row per protein (AlphaFold has one model per sequence, so repeat depositions are collapsed — “×N” marks how many structures of that protein exist; the worst is shown). Each row joins what AlphaFold claimed (blue, pLDDT/100) to what the experiment showed (red, TM-score) — the longer the bar, the bigger the miss. Hover a dot to preview its Cα-deviation ribbon; click to open the full entry.

How to read this

Every point is one experimental protein structure. The horizontal axis is AlphaFold's own confidence in its prediction (mean pLDDT, 0–100). The vertical axis is how well that blind prediction actually matches the experiment (TM-score, 0–1; above 0.5 means the same fold, above 0.9 near-identical). Marker size grows with the FRAUD score (confidence-weighted error).

Cutoff: the shaded red box is the "confidently wrong" zone — AlphaFold was confident (mean pLDDT above 70) yet the fold is wrong (TM-score below 0.5). A predictor that had truly solved folding would leave that box empty.

Take-home: 4 of 77 structures (5.2%) are confidently wrong; median TM-score is 0.979.

Read moreShow less — how each metric is calculated

TM-score: how similar the two 3D shapes are overall, 0–1 (a random pair scores ~0.17, an identical fold ~1). Length-normalised so large and small proteins compare fairly. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) — dᵢ is the gap between the i-th aligned Cα atoms after best-fit superposition, and d₀ = 1.24(L−15)^⅓ − 1.8 sets the distance scale for length L. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264; computed with TM-align, doi:10.1093/nar/gki524.

pLDDT: AlphaFold's own confidence in each residue, 0–100 (higher = surer). It is the model predicting its own accuracy before ever seeing the experiment. We plot the per-structure mean. pLDDT = (1/N) Σᵢ pLDDTᵢ, where pLDDTᵢ is the network's confidence output for residue i.

FRAUD score: the headline number — how wrong the prediction was, weighted by how confident AlphaFold was, so a big error it was sure about counts most. FRAUD = (1/N) Σᵢ (pLDDTᵢ/100) · min(Δᵢ,15)/15, where Δᵢ is residue i's Cα distance from the experiment (Å) after superposition, capped at 15 Å. Runs 0 (perfect) to 1.

Novelty: how little AlphaFold had to go on — 100 minus the highest sequence identity between this protein and any structure released before the 2018-04-30 training cutoff. High = genuinely unseen (100% = nothing similar was ever in the training set). novelty = 100 − maxₚ identity(s, p) over pre-cutoff PDB chains p, with identity = 100 × (matching aligned residues)/(alignment length), from an MMseqs2 search.

Homology: the point colour. High novelty (above 70%) flags the sequence as novel (amber) — AlphaFold had no close template; otherwise a pre-cutoff homolog existed (blue) it could have learned the fold from. novel ⇔ novelty > 70%.

What the metrics mean

TM-score (0–1): overall fold match — above 0.5 is the same fold, above 0.9 near-identical. Cα-RMSD (Å): average backbone distance after best-fit superposition — lower is better (under 2 Å is excellent). lDDT (0–1): local accuracy measured without superposition — above 0.8 is good. Each bar counts how many structures fall in that range.

Take-home: median TM-score 0.979 — most predictions match the experimental fold well, with a long tail that do not.

Read moreShow less — how each metric is calculated

TM-score: overall shape match, 0–1 (above 0.5 = same fold, above 0.9 near-identical), length-normalised. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) with dᵢ the aligned-Cα gap after superposition and d₀ = 1.24(L−15)^⅓ − 1.8. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264.

Cα-RMSD: the average straight-line distance between matching backbone Cα atoms once the two structures are best-fit superposed (Å; lower is better, under ~2 Å excellent). RMSD = √( (1/N) Σᵢ |Pᵢ − (R·Qᵢ + t)|² ), with Pᵢ/Qᵢ the experimental/model Cα coordinates and R,t the rotation and translation from Kabsch superposition.

lDDT: local accuracy with no superposition — the fraction of short-range inter-residue distances the model reproduces, so it is not fooled by a single wrongly-placed domain. lDDT = (1/N) Σᵢ ¼ Σ_t 1[ |d_exp − d_model| < t ] over thresholds t ∈ {0.5, 1, 2, 4} Å and all residue pairs within 15 Å.

Trend over time

The blue line is the mean TM-score of the structures by their deposit month; the red bars count how many were confidently wrong. Binning by deposit date (rather than by when we processed them) gives an even timeline, tracking whether AlphaFold's accuracy on newly-deposited structures is holding steady, improving, or slipping as the PDB keeps growing. Months with fewer than 5 structures are omitted so each point is a meaningful average. A dashed trend line is drawn only if the change over time is statistically significant.

Read moreShow less — how each metric is calculated

Mean TM-score: the month's average shape-match score. TM̄ = (1/M) Σ TM over the M structures deposited that month.

Trend line: a monotonic-trend test over the monthly means — Mann-Kendall (Kendall's τ) for significance, with a robust Theil-Sen slope for the line. Rank-based, so it tolerates the skewed TM distribution and noisy low-count months. Shown only when p < 0.05; otherwise a note reports that no significant trend was found.

Confidently wrong: the count that were both confident and wrong. count( mean pLDDT > 70 AND TM < 0.5 ).

Matched structures

PDBUniProtProteinMethodÅDeposited Novelty %pLDDTTMlDDTGDT_TSRMSDFRAUDFlag
6ZE9_A Q96JC1 Vam6/Vps39-like protein X-ray 2.90 2020-06-16 100.00 novel 81.24 0.36 0.39 10.98 13.64 0.56 wrong
6UP6_A Q9Y371 Endophilin-B1 EM 9.00 2019-10-16 7.20 89.77 0.70 0.65 19.34 14.62 0.49 ok
6KBE_G P0CG48 Polyubiquitin-C X-ray 2.34 2019-06-24 0.70 89.92 0.62 0.94 23.50 7.40 0.40 ok
6Z3R_A Q96Q15 Serine/threonine-protein kinase SMG1,Serin EM 2.97 2020-05-21 71.50 novel 84.18 0.50 0.79 38.59 14.79 0.27 wrong
6TRN_C P43358 MAGE-A4 peptide (amino acids 230-239) vari X-ray 1.35 2019-12-19 90.90 0.40 0.65 37.50 4.32 0.23 wrong
6WWZ_A P09471 Guanine nucleotide-binding protein G(o) su EM 3.34 2020-05-09 94.50 0.75 0.23 ok
6TRO_C P43358 MAGE-A4 peptide (amino acids 230-239) X-ray 3.00 2019-12-19 90.90 0.22 0.68 40.00 4.16 0.23 wrong
6P8I_A P11717 Cation-independent mannose-6-phosphate rec X-ray 2.54 2019-06-07 22.40 78.71 0.89 0.84 57.13 3.96 0.16 ok
6X5J_B P00740 Coagulation factor IX X-ray 2.51 2020-05-26 80.31 0.85 0.12 ok
6X5P_B P00740 Coagulation factor IX X-ray 2.00 2020-05-26 80.31 0.85 0.12 ok
6X5L_B P00740 Coagulation factor IX X-ray 2.25 2020-05-26 80.31 0.86 0.11 ok
6WWZ_C P78556 C-C motif chemokine 20 EM 3.34 2020-05-09 88.69 0.88 0.10 ok
6PV3_A P08047 Transcription factor Sp1 NMR 2019-07-19 13.40 83.05 0.71 0.85 75.00 2.29 0.10 ok
6UCO_A P08047 Transcription factor Sp1 NMR 2019-09-17 9.70 82.96 0.79 0.87 82.26 2.22 0.08 ok
6PV2_A P08047 Transcription factor Sp1 NMR 2019-07-19 9.70 83.05 0.76 0.86 84.17 2.20 0.08 ok
6UCP_A P08047 Transcription factor Sp1 NMR 2019-09-17 3.30 82.96 0.71 0.87 79.03 1.78 0.08 ok
6PV0_A P08047 Transcription factor Sp1 NMR 2019-07-19 9.70 82.96 0.80 0.88 85.48 1.97 0.07 ok
6TRO_A Q53Z42 MHC class I antigen X-ray 3.00 2019-12-19 0.00 96.24 0.96 0.96 87.27 1.26 0.07 ok
6K9P_G P0CG47 Ubiquitin X-ray 2.05 2019-06-17 0.00 94.12 0.95 0.96 89.14 2.12 0.06 ok
6LRC_A Q8N884 Cyclic GMP-AMP synthase X-ray 1.83 2020-01-15 0.00 94.05 0.97 0.89 89.52 1.64 0.06 ok
6LRK_A Q8N884 Cyclic GMP-AMP synthase X-ray 2.25 2020-01-16 0.00 93.86 0.97 0.89 89.99 1.57 0.06 ok
6LRL_A Q8N884 Cyclic GMP-AMP synthase X-ray 2.65 2020-01-16 0.00 94.62 0.98 0.91 92.20 1.52 0.06 ok
6TRN_A Q53Z42 MHC class I antigen X-ray 1.35 2019-12-19 0.00 96.16 0.97 0.96 89.32 1.12 0.06 ok
6KA7_C P0DOX5 Immunoglobulin gamma-1 heavy chain X-ray 3.00 2019-06-21 0.00 96.17 0.97 0.94 91.54 1.01 0.06 ok
6YN9_A Q9UDY8 Mucosa-associated lymphoid tissue lymphoma X-ray 2.56 2020-04-11 79.44 0.93 0.05 ok
6PV1_A P08047 Transcription factor Sp1 NMR 2019-07-19 9.70 82.96 0.87 0.90 91.94 1.51 0.05 ok
6LRE_A Q8N884 Cyclic GMP-AMP synthase X-ray 2.65 2020-01-16 0.00 94.67 0.98 0.90 92.78 1.34 0.05 ok
6LRI_A Q8N884 Cyclic GMP-AMP synthase X-ray 2.50 2020-01-16 0.00 94.41 0.98 0.92 93.11 1.30 0.05 ok
6LRJ_A Q8N884 Cyclic GMP-AMP synthase X-ray 3.00 2020-01-16 0.00 94.60 0.97 0.91 93.07 1.32 0.05 ok
6Z3R_B Q8ND04 Protein SMG8 EM 2.97 2020-05-21 73.88 0.94 0.04 ok
6OQ3_A O00522 Krev interaction trapped protein 1 X-ray 1.85 2019-04-25 0.00 91.39 0.98 0.93 94.44 1.07 0.04 ok
6KD5_A Q9BYE2 Transmembrane protease serine 13 X-ray 2.60 2019-06-30 100.00 novel 89.07 0.96 0.92 93.51 1.16 0.04 ok
6YN8_A Q9UDY8 Mucosa-associated lymphoid tissue lymphoma X-ray 3.05 2020-04-11 79.44 0.95 0.04 ok
6OQ4_A O00522 Krev interaction trapped protein 1 X-ray 1.75 2019-04-25 0.00 91.39 0.98 0.94 94.52 0.98 0.04 ok
6KD5_B Q9BYE2 Transmembrane protease serine 13 X-ray 2.60 2019-06-30 55.70 90.89 0.97 0.93 95.59 1.48 0.04 ok
6TX0_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 2.01 2020-01-13 0.40 96.15 0.99 0.98 98.43 0.89 0.03 ok
6TXC_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 2.84 2020-01-14 0.20 96.00 0.99 0.98 98.18 0.87 0.03 ok
6TRO_B P61769 Beta-2-microglobulin X-ray 3.00 2019-12-19 0.00 96.78 0.98 0.97 97.75 0.57 0.03 ok
6Z3R_C Q9H0W8 Protein SMG9 EM 2.97 2020-05-21 70.56 0.96 0.03 ok
6TXA_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 2.85 2020-01-13 0.40 96.09 1.00 0.99 99.17 0.51 0.03 ok
6TXF_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 2.25 2020-01-14 0.40 96.12 1.00 0.98 98.90 0.51 0.03 ok
6TXE_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 3.19 2020-01-14 0.20 96.09 1.00 0.99 99.17 0.48 0.03 ok
6TRN_B P61769 Beta-2-microglobulin X-ray 1.35 2019-12-19 0.00 97.09 0.98 0.98 99.23 0.45 0.03 ok
6UYC_A Q15562 Transcriptional enhancer factor TEF-4 X-ray 1.66 2019-11-12 70.75 0.97 0.02 ok
6OQ3_B P61224 Ras-related protein Rap-1b X-ray 1.85 2019-04-25 0.00 95.86 0.99 0.97 98.93 0.65 0.02 ok
6OQ4_B P61224 Ras-related protein Rap-1b X-ray 1.75 2019-04-25 0.00 95.86 0.99 0.97 98.78 0.65 0.02 ok
6SY1_A Q96HY7 Probable 2-oxoglutarate dehydrogenase E1 c X-ray 1.87 2019-09-26 60.10 97.70 1.00 0.99 98.69 0.47 0.02 ok
6X4M_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.50 2020-05-22 98.06 0.98 0.02 ok
6YSQ_E P0C0L5 Complement C4 gamma chain X-ray 3.30 2020-04-23 83.38 0.98 0.02 ok
6UYB_A Q15562 Transcriptional enhancer factor TEF-4 X-ray 1.54 2019-11-12 70.75 0.97 0.02 ok
6X3Y_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.40 2020-05-21 98.06 0.98 0.02 ok
6X5J_A P00740 Coagulation factor IX X-ray 2.51 2020-05-26 80.31 0.98 0.02 ok
7C3N_A P52333 Tyrosine-protein kinase JAK3 X-ray 1.98 2020-05-13 85.69 0.98 0.02 ok
6X5L_A P00740 Coagulation factor IX X-ray 2.25 2020-05-26 80.31 0.98 0.02 ok
6X5P_A P00740 Coagulation factor IX X-ray 2.00 2020-05-26 80.31 0.98 0.02 ok
6WLX_A O96013 Serine/threonine-protein kinase PAK 4 X-ray 2.20 2020-04-20 70.06 0.98 0.01 ok
6WLY_A O96013 Serine/threonine-protein kinase PAK 4 X-ray 1.90 2020-04-20 70.06 0.98 0.01 ok
6XZ8_A P19099 Cytochrome P450 11B2, mitochondrial X-ray 3.00 2020-02-03 89.75 0.99 0.01 ok
6XZ9_A P19099 Cytochrome P450 11B2, mitochondrial X-ray 2.77 2020-02-03 89.75 0.99 0.01 ok
6X4N_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.51 2020-05-22 98.06 0.99 0.01 ok
6X4Q_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.80 2020-05-22 98.06 0.99 0.01 ok
6VQL_A Q9NWZ3 Interleukin-1 receptor-associated kinase 4 X-ray 2.07 2020-02-05 83.94 0.99 0.01 ok
6YSQ_C P0C0L4 Complement C4-A alpha chain X-ray 3.30 2020-04-23 83.44 0.99 0.01 ok
6YSQ_A P0C0L5 Complement C4 beta chain X-ray 3.30 2020-04-23 83.38 0.99 0.01 ok
6X4P_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.50 2020-05-22 98.06 0.99 0.01 ok
6WQ8_A P00918 Carbonic anhydrase 2 X-ray 1.41 2020-04-28 97.38 1.00 0.00 ok
6XU1_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 2.20 2020-01-17 88.19 1.00 0.00 ok
6X4O_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.50 2020-05-22 98.06 1.00 0.00 ok
6YOM_A Q9Y3Z3 Deoxynucleoside triphosphate triphosphohyd X-ray 3.25 2020-04-14 88.19 1.00 0.00 ok
6X3R_A P62937 Peptidyl-prolyl cis-trans isomerase A X-ray 1.80 2020-05-21 98.06 1.00 0.00 ok
6WQ9_A P00918 Carbonic anhydrase 2 X-ray 1.30 2020-04-28 97.38 1.00 0.00 ok
6WQ7_A P00918 Carbonic anhydrase 2 X-ray 1.30 2020-04-28 97.38 1.00 0.00 ok
6WQ5_A P00918 Carbonic anhydrase 2 X-ray 1.30 2020-04-28 97.38 1.00 0.00 ok
6WQ4_A P00918 Carbonic anhydrase 2 X-ray 1.35 2020-04-28 97.38 1.00 0.00 ok
6Z6U_1 P02794 Ferritin heavy chain EM 1.25 2020-05-29 95.31 1.00 0.00 ok
6Z9E_1 P02794 Ferritin heavy chain EM 1.55 2020-06-03 95.31 1.00 0.00 ok
6Z9F_1 P02794 Ferritin heavy chain EM 1.56 2020-06-03 95.31 1.00 0.00 ok

Click a column header to sort. FRAUD = confidence-weighted error; a structure is flagged confidently wrong when mean pLDDT > 70 yet TM-score < 0.5.