Live Stats, next update: Wed 02 Sep
Human PDBs Analysed
Confidently Wrong
Novel + Confidently Wrong
DB size
Visitors
Full statistics →
New PDB Depositions vs. Their Blind AlphaFold Predictions — A Running Test of “Is Folding Solved?”

Release week 2020-01-22

69
structures analysed (65 full · 94.2%)
11.4%
confidently wrong
811.6%
novel sequences
00.0%
novel & wrong
0.967
median TM-score

Zoomed into the red "confidently wrong" box above (pLDDT ≥ 70, TM < 0.5) — the structures AlphaFold got confidently wrong.

The worst offenders ranked, one row per protein (AlphaFold has one model per sequence, so repeat depositions are collapsed — “×N” marks how many structures of that protein exist; the worst is shown). Each row joins what AlphaFold claimed (blue, pLDDT/100) to what the experiment showed (red, TM-score) — the longer the bar, the bigger the miss. Hover a dot to preview its Cα-deviation ribbon; click to open the full entry.

How to read this

Every point is one experimental protein structure. The horizontal axis is AlphaFold's own confidence in its prediction (mean pLDDT, 0–100). The vertical axis is how well that blind prediction actually matches the experiment (TM-score, 0–1; above 0.5 means the same fold, above 0.9 near-identical). Marker size grows with the FRAUD score (confidence-weighted error).

Cutoff: the shaded red box is the "confidently wrong" zone — AlphaFold was confident (mean pLDDT above 70) yet the fold is wrong (TM-score below 0.5). A predictor that had truly solved folding would leave that box empty.

Take-home: 1 of 69 structures (1.4%) are confidently wrong; median TM-score is 0.967.

Read moreShow less — how each metric is calculated

TM-score: how similar the two 3D shapes are overall, 0–1 (a random pair scores ~0.17, an identical fold ~1). Length-normalised so large and small proteins compare fairly. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) — dᵢ is the gap between the i-th aligned Cα atoms after best-fit superposition, and d₀ = 1.24(L−15)^⅓ − 1.8 sets the distance scale for length L. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264; computed with TM-align, doi:10.1093/nar/gki524.

pLDDT: AlphaFold's own confidence in each residue, 0–100 (higher = surer). It is the model predicting its own accuracy before ever seeing the experiment. We plot the per-structure mean. pLDDT = (1/N) Σᵢ pLDDTᵢ, where pLDDTᵢ is the network's confidence output for residue i.

FRAUD score: the headline number — how wrong the prediction was, weighted by how confident AlphaFold was, so a big error it was sure about counts most. FRAUD = (1/N) Σᵢ (pLDDTᵢ/100) · min(Δᵢ,15)/15, where Δᵢ is residue i's Cα distance from the experiment (Å) after superposition, capped at 15 Å. Runs 0 (perfect) to 1.

Novelty: how little AlphaFold had to go on — 100 minus the highest sequence identity between this protein and any structure released before the 2018-04-30 training cutoff. High = genuinely unseen (100% = nothing similar was ever in the training set). novelty = 100 − maxₚ identity(s, p) over pre-cutoff PDB chains p, with identity = 100 × (matching aligned residues)/(alignment length), from an MMseqs2 search.

Homology: the point colour. High novelty (above 70%) flags the sequence as novel (amber) — AlphaFold had no close template; otherwise a pre-cutoff homolog existed (blue) it could have learned the fold from. novel ⇔ novelty > 70%.

What the metrics mean

TM-score (0–1): overall fold match — above 0.5 is the same fold, above 0.9 near-identical. Cα-RMSD (Å): average backbone distance after best-fit superposition — lower is better (under 2 Å is excellent). lDDT (0–1): local accuracy measured without superposition — above 0.8 is good. Each bar counts how many structures fall in that range.

Take-home: median TM-score 0.967 — most predictions match the experimental fold well, with a long tail that do not.

Read moreShow less — how each metric is calculated

TM-score: overall shape match, 0–1 (above 0.5 = same fold, above 0.9 near-identical), length-normalised. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) with dᵢ the aligned-Cα gap after superposition and d₀ = 1.24(L−15)^⅓ − 1.8. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264.

Cα-RMSD: the average straight-line distance between matching backbone Cα atoms once the two structures are best-fit superposed (Å; lower is better, under ~2 Å excellent). RMSD = √( (1/N) Σᵢ |Pᵢ − (R·Qᵢ + t)|² ), with Pᵢ/Qᵢ the experimental/model Cα coordinates and R,t the rotation and translation from Kabsch superposition.

lDDT: local accuracy with no superposition — the fraction of short-range inter-residue distances the model reproduces, so it is not fooled by a single wrongly-placed domain. lDDT = (1/N) Σᵢ ¼ Σ_t 1[ |d_exp − d_model| < t ] over thresholds t ∈ {0.5, 1, 2, 4} Å and all residue pairs within 15 Å.

Trend over time

The blue line is the mean TM-score of the structures by their deposit month; the red bars count how many were confidently wrong. Binning by deposit date (rather than by when we processed them) gives an even timeline, tracking whether AlphaFold's accuracy on newly-deposited structures is holding steady, improving, or slipping as the PDB keeps growing. Months with fewer than 5 structures are omitted so each point is a meaningful average. A dashed trend line is drawn only if the change over time is statistically significant.

Read moreShow less — how each metric is calculated

Mean TM-score: the month's average shape-match score. TM̄ = (1/M) Σ TM over the M structures deposited that month.

Trend line: a monotonic-trend test over the monthly means — Mann-Kendall (Kendall's τ) for significance, with a robust Theil-Sen slope for the line. Rank-based, so it tolerates the skewed TM distribution and noisy low-count months. Shown only when p < 0.05; otherwise a note reports that no significant trend was found.

Confidently wrong: the count that were both confident and wrong. count( mean pLDDT > 70 AND TM < 0.5 ).

Matched structures

PDBUniProtProteinMethodÅDeposited Novelty %pLDDTTMlDDTGDT_TSRMSDFRAUDFlag
6KOR_A O60506 Heterogeneous nuclear ribonucleoprotein Q X-ray 2.60 2019-08-12 0.00 88.57 0.55 0.90 7.78 13.93 0.64 ok
6V63_Y P63261 Actin, cytoplasmic 1 X-ray 2.02 2019-12-04 4.40 96.36 0.22 0.64 18.42 8.55 0.51 wrong
6U6N_C Q15848 Adiponectin X-ray 2.15 2019-08-30 2.90 95.85 0.85 0.73 48.60 5.78 0.26 ok
6S9I_B O75643 U5 small nuclear ribonucleoprotein 200 kDa X-ray 2.60 2019-07-13 0.20 89.55 0.89 0.93 37.68 4.70 0.26 ok
6S8Q_B O75643 U5 small nuclear ribonucleoprotein 200 kDa X-ray 2.39 2019-07-10 0.10 89.45 0.89 0.95 37.87 4.65 0.26 ok
6S9K_B P42575 Caspase-2 X-ray 1.60 2019-07-15 100.00 novel 27.12 0.42 0.29 14.77 12.78 0.19 ok
6TM1_C Q96BY6 Dedicator of cytokinesis protein 10 X-ray 3.71 2019-12-03 34.50 89.40 0.91 0.80 61.86 2.81 0.14 ok
6KHZ_A Q13501 Sequestosome-1 X-ray 2.80 2019-07-16 59.10 90.75 0.86 0.93 67.35 3.60 0.14 ok
6JFV_B P13647 Keratin, type II cytoskeletal 5 X-ray 2.60 2019-02-12 0.00 94.82 0.91 0.98 70.79 2.09 0.11 ok
6PNX_A P22607 Fibroblast growth factor receptor 3 X-ray 2.20 2019-07-03 1.00 88.09 0.93 0.89 70.96 3.79 0.11 ok
6S8O_B O75643 U5 small nuclear ribonucleoprotein 200 kDa X-ray 3.17 2019-07-10 0.20 89.43 0.98 0.89 72.94 2.04 0.10 ok
6NMY_A P32927 Cytokine receptor common subunit beta X-ray 3.30 2019-01-13 0.00 91.74 0.92 0.95 74.07 1.69 0.10 ok
6TM1_A P60763 Ras-related C3 botulinum toxin substrate 3 X-ray 3.71 2019-12-03 0.60 96.63 0.92 0.82 80.79 2.29 0.09 ok
6J6X_A Q7Z6K3 Protein prenyltransferase alpha subunit re X-ray 2.96 2019-01-16 73.20 novel 94.92 0.94 0.90 78.69 1.97 0.09 ok
6TKZ_C P60953 Cell division control protein 42 homolog X-ray 2.64 2019-11-29 0.00 96.23 0.92 0.89 82.77 2.37 0.09 ok
6TKY_C P60953 Cell division control protein 42 homolog X-ray 2.55 2019-11-29 0.00 96.23 0.92 0.89 82.91 2.31 0.09 ok
6S8Q_J Q6P2Q9 Pre-mRNA-processing-splicing factor 8 X-ray 2.39 2019-07-10 0.00 78.98 0.95 0.91 75.76 3.27 0.09 ok
6NIX_C P02458 Type II Collagen X-ray 2.10 2019-01-01 41.55 0.36 0.85 51.92 3.28 0.08 ok
6SY2_A P51532 Transcription activator BRG1 NMR 2019-09-27 57.90 70.96 0.71 0.80 75.00 2.44 0.08 ok
6JFV_A P02533 Keratin, type I cytoskeletal 14 X-ray 2.60 2019-02-12 1.10 95.14 0.90 0.98 83.60 1.38 0.08 ok
6SAD_C P42575 Caspase-2 X-ray 2.75 2019-07-16 100.00 novel 30.33 0.19 0.73 48.68 4.29 0.07 ok
6NMY_I P08700 Interleukin-3 X-ray 3.30 2019-01-13 0.00 92.04 0.91 0.86 87.04 1.92 0.07 ok
6LK0_A Q15645 Pachytene checkpoint protein 2 homolog X-ray 2.60 2019-12-17 0.30 92.63 0.97 0.93 84.75 1.64 0.07 ok
6NOW_A Q5JTZ9 Alanine--tRNA ligase, mitochondrial X-ray 4.10 2019-01-16 68.80 92.40 0.94 0.91 84.25 1.28 0.07 ok
6NQ5_B P68871 Hemoglobin subunit beta X-ray 1.85 2019-01-19 0.00 97.51 0.93 0.88 87.41 1.54 0.07 ok
6TM1_B Q96BY6 Dedicator of cytokinesis protein 10 X-ray 3.71 2019-12-03 34.50 89.19 0.97 0.85 85.56 1.37 0.07 ok
6NIX_B P01911 HLA class II histocompatibility antigen, D X-ray 2.10 2019-01-01 0.00 96.16 0.96 0.96 91.53 2.00 0.06 ok
6S9I_J Q6P2Q9 Pre-mRNA-processing-splicing factor 8 X-ray 2.60 2019-07-13 0.00 79.71 0.96 0.93 85.42 2.69 0.06 ok
6QNX_A Q8N3U4 Cohesin subunit SA-2 X-ray 2.70 2019-02-12 0.00 93.26 0.99 0.95 89.92 1.24 0.06 ok
6TKY_B Q96BY6 Dedicator of cytokinesis protein 10 X-ray 2.55 2019-11-29 34.50 89.12 0.98 0.97 87.98 1.15 0.06 ok
6NMY_B P32927 Cytokine receptor common subunit beta X-ray 3.30 2019-01-13 0.00 91.45 0.96 0.97 89.03 1.06 0.06 ok
6J7F_C O15498 Synaptobrevin homolog YKT6 X-ray 2.88 2019-01-18 5.20 91.76 0.96 0.93 90.77 1.80 0.06 ok
6J7F_A Q7Z6K3 Protein prenyltransferase alpha subunit re X-ray 2.88 2019-01-18 73.20 novel 93.91 0.97 0.90 91.93 1.75 0.05 ok
6J6X_B P53611 Geranylgeranyl transferase type-2 subunit X-ray 2.96 2019-01-16 4.30 97.18 0.97 0.92 92.79 1.73 0.05 ok
6J74_A Q7Z6K3 Protein prenyltransferase alpha subunit re X-ray 3.21 2019-01-16 73.20 novel 94.93 0.97 0.91 93.39 1.52 0.05 ok
6NIX_A P01903 HLA class II histocompatibility antigen, D X-ray 2.10 2019-01-01 0.00 95.73 0.97 0.97 91.57 0.94 0.05 ok
6TKZ_A Q96BY6 Dedicator of cytokinesis protein 10 X-ray 2.64 2019-11-29 34.50 88.98 0.98 0.97 91.91 0.98 0.05 ok
6QNX_B O60216 64-kDa C-terminal product X-ray 2.70 2019-02-12 0.00 93.82 0.93 0.96 93.24 0.88 0.05 ok
6KGJ_A Q96HA8 Protein N-terminal glutamine amidohydrolas X-ray 1.80 2019-07-11 0.50 94.90 0.97 0.98 92.45 1.55 0.05 ok
6TU9_A Q01973 Inactive tyrosine-protein kinase transmemb X-ray 1.94 2020-01-04 33.90 88.29 0.97 0.95 90.36 1.12 0.05 ok
6TKY_A Q96BY6 Dedicator of cytokinesis protein 10 X-ray 2.55 2019-11-29 34.50 88.98 0.98 0.97 92.33 0.95 0.05 ok
6J7X_A Q7Z6K3 Protein prenyltransferase alpha subunit re X-ray 2.75 2019-01-18 73.20 novel 94.66 0.97 0.93 94.75 1.45 0.05 ok
6NR0_A Q8IXJ6 NAD-dependent protein deacetylase sirtuin- X-ray 2.45 2019-01-22 0.00 93.50 0.98 0.96 94.90 0.93 0.04 ok
6J6V_A Q9ULV5 Heat shock factor protein 4 X-ray 1.20 2019-01-16 22.90 92.95 0.95 0.92 94.41 0.98 0.04 ok
6J6W_A Q9ULV5 Heat shock factor protein 4 X-ray 1.69 2019-01-16 23.90 92.95 0.95 0.92 93.88 0.96 0.04 ok
6NMY_F P26951 Interleukin-3 receptor subunit alpha X-ray 3.30 2019-01-13 0.40 94.31 0.98 0.95 96.11 0.88 0.04 ok
6JJB_A O60885 Bromodomain-containing protein 4 X-ray 1.51 2019-02-25 0.00 94.57 0.97 0.95 94.80 0.91 0.04 ok
6SAD_A P61981 14-3-3 protein gamma X-ray 2.75 2019-07-16 0.00 96.51 0.98 0.96 95.60 0.80 0.04 ok
6JJ3_A O60885 Bromodomain-containing protein 4 X-ray 1.72 2019-02-25 0.00 94.57 0.97 0.95 95.00 0.88 0.04 ok
6J7X_C O15498 Synaptobrevin homolog YKT6 X-ray 2.75 2019-01-18 5.10 92.40 0.98 0.94 95.83 0.88 0.04 ok
6IQN_A P04629 High affinity nerve growth factor receptor X-ray 2.54 2018-11-08 0.00 89.15 0.98 0.95 95.22 1.06 0.03 ok
6J74_C O15498 Synaptobrevin homolog YKT6 X-ray 3.21 2019-01-16 5.10 92.40 0.98 0.94 97.92 0.64 0.03 ok
6S9K_A P61981 14-3-3 protein gamma X-ray 1.60 2019-07-15 0.00 96.74 0.99 0.98 97.48 0.70 0.03 ok
6J7F_B P53611 Geranylgeranyl transferase type-2 subunit X-ray 2.88 2019-01-18 4.30 96.95 0.99 0.97 97.19 0.83 0.03 ok
6J74_B P53611 Geranylgeranyl transferase type-2 subunit X-ray 3.21 2019-01-16 4.30 97.03 0.99 0.97 97.64 0.73 0.03 ok
6J7X_B P53611 Geranylgeranyl transferase type-2 subunit X-ray 2.75 2019-01-18 4.30 97.10 0.99 0.97 97.63 0.72 0.03 ok
6J8O_B Q7L8A9 Tubulinyl-Tyr carboxypeptidase 1 X-ray 1.85 2019-01-20 100.00 novel 96.38 0.99 0.99 97.98 0.55 0.03 ok
6NPR_B P61769 Beta-2-microglobulin X-ray 2.37 2019-01-18 0.00 96.78 0.98 0.98 99.00 0.49 0.03 ok
6NQ5_A P69905 Hemoglobin subunit alpha X-ray 1.85 2019-01-19 0.00 98.34 0.99 0.97 99.29 0.47 0.03 ok
6RUA_A P06276 Cholinesterase X-ray 2.75 2019-05-27 0.00 96.92 1.00 0.98 98.29 0.59 0.02 ok
6J8O_A Q8N300 Small vasohibin-binding protein X-ray 1.85 2019-01-20 100.00 novel 96.18 0.92 1.00 100.00 0.40 0.02 ok
6PRX_A O15382 Branched-chain-amino-acid aminotransferase X-ray 3.25 2019-07-11 0.60 97.99 1.00 0.98 99.43 0.52 0.02 ok
6H05_A P36957 Dihydrolipoyllysine-residue succinyltransf EM 2.90 2018-07-06 76.44 0.97 0.02 ok
6U4T_A P00918 Carbonic anhydrase 2 X-ray 1.36 2019-08-26 0.00 97.89 1.00 0.99 99.61 0.37 0.02 ok
6U4Q_A P00918 Carbonic anhydrase 2 X-ray 1.31 2019-08-26 0.00 97.89 1.00 0.99 99.61 0.35 0.02 ok
6U66_A Q15848 Adiponectin X-ray 0.99 2019-08-29 0.80 95.65 0.99 0.99 99.26 0.41 0.02 ok
6HZN_A Q9UL01 Dermatan-sulfate epimerase X-ray 2.41 2018-10-23 86.56 0.99 0.01 ok
6USC_A Q8WWA0 Intelectin-1 X-ray 1.59 2019-10-25 92.44 1.00 0.00 ok
6V63_A Q86TU7 Actin-histidine N-methyltransferase X-ray 2.02 2019-12-04 86.38 1.00 0.00 ok

Click a column header to sort. FRAUD = confidence-weighted error; a structure is flagged confidently wrong when mean pLDDT > 70 yet TM-score < 0.5.