Live Stats, next update: Wed 02 Sep
Human PDBs Analysed
Confidently Wrong
Novel + Confidently Wrong
DB size
Visitors
Full statistics →
New PDB Depositions vs. Their Blind AlphaFold Predictions — A Running Test of “Is Folding Solved?”

Release week 2022-03-09

81
structures analysed (8 full · 9.9%)
89.9%
confidently wrong
33.7%
novel sequences
33.7%
novel & wrong
0.94
median TM-score

Zoomed into the red "confidently wrong" box above (pLDDT ≥ 70, TM < 0.5) — the structures AlphaFold got confidently wrong.

The worst offenders ranked, one row per protein (AlphaFold has one model per sequence, so repeat depositions are collapsed — “×N” marks how many structures of that protein exist; the worst is shown). Each row joins what AlphaFold claimed (blue, pLDDT/100) to what the experiment showed (red, TM-score) — the longer the bar, the bigger the miss. Hover a dot to preview its Cα-deviation ribbon; click to open the full entry.

How to read this

Every point is one experimental protein structure. The horizontal axis is AlphaFold's own confidence in its prediction (mean pLDDT, 0–100). The vertical axis is how well that blind prediction actually matches the experiment (TM-score, 0–1; above 0.5 means the same fold, above 0.9 near-identical). Marker size grows with the FRAUD score (confidence-weighted error).

Cutoff: the shaded red box is the "confidently wrong" zone — AlphaFold was confident (mean pLDDT above 70) yet the fold is wrong (TM-score below 0.5). A predictor that had truly solved folding would leave that box empty.

Take-home: 8 of 81 structures (9.9%) are confidently wrong; median TM-score is 0.94.

Read moreShow less — how each metric is calculated

TM-score: how similar the two 3D shapes are overall, 0–1 (a random pair scores ~0.17, an identical fold ~1). Length-normalised so large and small proteins compare fairly. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) — dᵢ is the gap between the i-th aligned Cα atoms after best-fit superposition, and d₀ = 1.24(L−15)^⅓ − 1.8 sets the distance scale for length L. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264; computed with TM-align, doi:10.1093/nar/gki524.

pLDDT: AlphaFold's own confidence in each residue, 0–100 (higher = surer). It is the model predicting its own accuracy before ever seeing the experiment. We plot the per-structure mean. pLDDT = (1/N) Σᵢ pLDDTᵢ, where pLDDTᵢ is the network's confidence output for residue i.

FRAUD score: the headline number — how wrong the prediction was, weighted by how confident AlphaFold was, so a big error it was sure about counts most. FRAUD = (1/N) Σᵢ (pLDDTᵢ/100) · min(Δᵢ,15)/15, where Δᵢ is residue i's Cα distance from the experiment (Å) after superposition, capped at 15 Å. Runs 0 (perfect) to 1.

Novelty: how little AlphaFold had to go on — 100 minus the highest sequence identity between this protein and any structure released before the 2018-04-30 training cutoff. High = genuinely unseen (100% = nothing similar was ever in the training set). novelty = 100 − maxₚ identity(s, p) over pre-cutoff PDB chains p, with identity = 100 × (matching aligned residues)/(alignment length), from an MMseqs2 search.

Homology: the point colour. High novelty (above 70%) flags the sequence as novel (amber) — AlphaFold had no close template; otherwise a pre-cutoff homolog existed (blue) it could have learned the fold from. novel ⇔ novelty > 70%.

What the metrics mean

TM-score (0–1): overall fold match — above 0.5 is the same fold, above 0.9 near-identical. Cα-RMSD (Å): average backbone distance after best-fit superposition — lower is better (under 2 Å is excellent). lDDT (0–1): local accuracy measured without superposition — above 0.8 is good. Each bar counts how many structures fall in that range.

Take-home: median TM-score 0.94 — most predictions match the experimental fold well, with a long tail that do not.

Read moreShow less — how each metric is calculated

TM-score: overall shape match, 0–1 (above 0.5 = same fold, above 0.9 near-identical), length-normalised. TM = (1/L) Σᵢ 1/(1 + (dᵢ/d₀)²) with dᵢ the aligned-Cα gap after superposition and d₀ = 1.24(L−15)^⅓ − 1.8. Ref: Zhang & Skolnick, Proteins 2004 doi:10.1002/prot.20264.

Cα-RMSD: the average straight-line distance between matching backbone Cα atoms once the two structures are best-fit superposed (Å; lower is better, under ~2 Å excellent). RMSD = √( (1/N) Σᵢ |Pᵢ − (R·Qᵢ + t)|² ), with Pᵢ/Qᵢ the experimental/model Cα coordinates and R,t the rotation and translation from Kabsch superposition.

lDDT: local accuracy with no superposition — the fraction of short-range inter-residue distances the model reproduces, so it is not fooled by a single wrongly-placed domain. lDDT = (1/N) Σᵢ ¼ Σ_t 1[ |d_exp − d_model| < t ] over thresholds t ∈ {0.5, 1, 2, 4} Å and all residue pairs within 15 Å.

Trend over time

The blue line is the mean TM-score of the structures by their deposit month; the red bars count how many were confidently wrong. Binning by deposit date (rather than by when we processed them) gives an even timeline, tracking whether AlphaFold's accuracy on newly-deposited structures is holding steady, improving, or slipping as the PDB keeps growing. Months with fewer than 5 structures are omitted so each point is a meaningful average. A dashed trend line is drawn only if the change over time is statistically significant.

Read moreShow less — how each metric is calculated

Mean TM-score: the month's average shape-match score. TM̄ = (1/M) Σ TM over the M structures deposited that month.

Trend line: a monotonic-trend test over the monthly means — Mann-Kendall (Kendall's τ) for significance, with a robust Theil-Sen slope for the line. Rank-based, so it tolerates the skewed TM distribution and noisy low-count months. Shown only when p < 0.05; otherwise a note reports that no significant trend was found.

Confidently wrong: the count that were both confident and wrong. count( mean pLDDT > 70 AND TM < 0.5 ).

Matched structures

PDBUniProtProteinMethodÅDeposited Novelty %pLDDTTMlDDTGDT_TSRMSDFRAUDFlag
7SAS_A Q9NUM4 Transmembrane protein 106B EM 3.70 2021-09-23 100.00 novel 94.39 0.18 0.56 0.56 27.50 0.88 wrong
7SAR_A Q9NUM4 Transmembrane protein 106B EM 3.20 2021-09-23 100.00 novel 94.39 0.17 0.56 0.56 27.49 0.87 wrong
7SAQ_A Q9NUM4 Transmembrane protein 106B EM 2.90 2021-09-23 100.00 novel 94.39 0.18 0.55 0.56 27.41 0.87 wrong
7QWG_A Q9NUM4 Transmembrane protein 106B EM 3.38 2022-01-25 75.88 0.18 0.63 wrong
7QVF_A Q9NUM4 Transmembrane protein 106B EM 3.64 2022-01-21 75.88 0.18 0.63 wrong
7QVC_A Q9NUM4 Transmembrane protein 106B EM 2.64 2022-01-21 75.88 0.18 0.63 wrong
7QWM_A Q9NUM4 Transmembrane protein 106B EM 2.76 2022-01-25 75.88 0.20 0.61 wrong
7QWL_A Q9NUM4 Transmembrane protein 106B EM 3.47 2022-01-25 75.88 0.21 0.60 wrong
7QWR_u P20290 Isoform 2 of Transcription factor BTF3 EM 2.90 2022-01-25 72.62 0.67 0.24 ok
7VTP_A Q96P20 NACHT, LRR and PYD domains-containing prot EM 3.23 2021-10-30 81.06 0.75 0.20 ok
7EBK_B P68431 H3K9me3K18ac X-ray 1.74 2021-03-09 0.00 62.57 0.36 0.89 42.50 6.02 0.18 ok
7T10_A P08754 Guanine nucleotide-binding protein G(i) su EM 2.50 2021-11-30 93.81 0.81 0.18 ok
7VKT_B P63096 Guanine nucleotide-binding protein G(i) su EM 2.90 2021-10-01 93.75 0.81 0.18 ok
7T10_C P59768 Guanine nucleotide-binding protein G(I)/G( EM 2.50 2021-11-30 89.56 0.80 0.18 ok
7T10_P P61278 Somatostatin-14 EM 2.50 2021-11-30 59.88 0.29 0.62 39.29 4.70 0.17 ok
7T11_A P08754 Guanine nucleotide-binding protein G(i) su EM 2.70 2021-11-30 93.81 0.82 0.17 ok
7VKT_D P59768 Guanine nucleotide-binding protein G(I)/G( EM 2.90 2021-10-01 89.56 0.82 0.16 ok
7T11_C P59768 Guanine nucleotide-binding protein G(I)/G( EM 2.70 2021-11-30 89.56 0.82 0.16 ok
7VQP_C Q15648 Mediator of RNA polymerase II transcriptio X-ray 1.94 2021-10-20 50.12 0.75 0.13 ok
7QWR_t Q13765 Nascent polypeptide-associated complex sub EM 2.90 2022-01-25 72.69 0.85 0.11 ok
7RYC_O P30559 Oxytocin receptor EM 2.90 2021-08-24 78.62 0.86 0.11 ok
7R71_D P0CG48 Ubiquitin X-ray 2.80 2021-06-24 88.62 0.88 0.10 ok
7WPA_D Q9BYF1 Angiotensin-converting enzyme 2 EM 2.77 2022-01-23 90.69 0.90 0.09 ok
7RBG_A P06746 DNA polymerase beta X-ray 1.90 2021-07-06 94.25 0.91 0.09 ok
7RBI_A P06746 DNA polymerase beta X-ray 1.93 2021-07-06 94.25 0.92 0.08 ok
7RBM_A P06746 DNA polymerase beta X-ray 2.21 2021-07-06 94.25 0.92 0.07 ok
7RBF_A P06746 DNA polymerase beta X-ray 1.84 2021-07-06 94.25 0.93 0.07 ok
7VKT_A Q15722 Leukotriene B4 receptor 1 EM 2.90 2021-10-01 81.38 0.92 0.06 ok
7DYN_C P18583 ARG-ARG-PHE-SEP-ARG-SEP-PRO-ILE-ARG-ARG X-ray 2.00 2021-01-22 33.59 0.34 0.74 52.50 3.07 0.06 ok
7RBK_A P06746 DNA polymerase beta X-ray 2.20 2021-07-06 94.25 0.94 0.06 ok
7RBH_A P06746 DNA polymerase beta X-ray 1.75 2021-07-06 94.25 0.94 0.06 ok
7RBJ_A P06746 DNA polymerase beta X-ray 1.91 2021-07-06 94.25 0.94 0.06 ok
7RBL_A P06746 DNA polymerase beta X-ray 1.98 2021-07-06 94.25 0.94 0.06 ok
7CIR_C P18583 ARG-ARG-PHE-SEP-ARG-SER-PRO-ILE-ARG-ARG X-ray 1.81 2020-07-08 33.59 0.32 0.56 52.50 2.74 0.06 ok
7RBE_A P06746 DNA polymerase beta X-ray 1.89 2021-07-06 94.25 0.94 0.06 ok
7RBN_A P06746 DNA polymerase beta X-ray 2.90 2021-07-06 94.25 0.94 0.06 ok
7LWE_A P41182 B-cell lymphoma 6 protein X-ray 1.17 2021-03-01 52.06 0.89 0.06 ok
7RBO_A P06746 DNA polymerase beta X-ray 2.96 2021-07-06 94.25 0.95 0.05 ok
7CIQ_C P18583 ARG-ARG-PHE-SER-ARG-SER-PRO-ILE-ARG-ARG X-ray 1.59 2020-07-08 33.59 0.30 0.71 62.50 2.38 0.05 ok
7R71_C P62837 Ubiquitin-conjugating enzyme E2 D2 X-ray 2.80 2021-06-24 96.50 0.95 0.05 ok
7LAW_R Q9Y5U5 Tumor necrosis factor receptor superfamily X-ray 2.75 2021-01-07 74.38 0.94 0.05 ok
7Q4Y_A Q8IVV7 Glucose-induced degradation protein 4 homo X-ray 3.08 2021-11-02 74.38 0.94 0.05 ok
7R11_D Q9BYF1 Angiotensin-converting enzyme 2 EM 3.50 2022-02-02 90.69 0.95 0.04 ok
7R0Z_D Q9BYF1 Angiotensin-converting enzyme 2 EM 3.50 2022-02-02 90.69 0.95 0.04 ok
7R12_D Q9BYF1 Angiotensin-converting enzyme 2 EM 3.30 2022-02-02 90.69 0.96 0.04 ok
7LWG_A P41182 B-cell lymphoma 6 protein X-ray 1.30 2021-03-01 52.06 0.92 0.04 ok
7VDU_A P24941 Cyclin-dependent kinase 2 X-ray 1.53 2021-09-07 88.44 0.96 0.04 ok
7LWF_A P41182 B-cell lymphoma 6 protein X-ray 1.22 2021-03-01 52.06 0.93 0.04 ok
7EO5_A P25440 Bromodomain-containing protein 2 X-ray 2.00 2021-04-21 64.06 0.94 0.04 ok
7ENZ_A P25440 Bromodomain-containing protein 2 X-ray 1.70 2021-04-21 64.06 0.94 0.04 ok
7WPC_D Q9BYF1 Angiotensin-converting enzyme 2 EM 2.57 2022-01-23 90.69 0.96 0.03 ok
7LAW_A Q9UNG2 Tumor necrosis factor ligand superfamily m X-ray 2.75 2021-01-07 86.75 0.96 0.03 ok
7WPB_D Q9BYF1 Angiotensin-converting enzyme 2 EM 2.79 2022-01-23 90.69 0.96 0.03 ok
7QWS_K Q8N0Z6 Tetratricopeptide repeat protein 5 EM 3.40 2022-01-25 93.38 0.97 0.03 ok
7VDS_A Q00535 Cyclin-dependent-like kinase 5, p25 X-ray 3.05 2021-09-07 91.56 0.97 0.02 ok
7VDP_A Q00535 Cyclin-dependent-like kinase 5 X-ray 2.09 2021-09-07 91.56 0.97 0.02 ok
7VDR_A Q00535 Cyclin-dependent-like kinase 5 X-ray 2.55 2021-09-07 91.56 0.97 0.02 ok
7VDQ_A Q00535 Cyclin-dependent-like kinase 5 X-ray 2.91 2021-09-07 91.56 0.97 0.02 ok
7CIR_B P61769 Beta-2-microglobulin X-ray 1.81 2020-07-08 94.06 0.98 0.02 ok
7RYC_C P62873 Guanine nucleotide-binding protein G(I)/G( EM 2.90 2021-08-24 97.06 0.98 0.02 ok
7DYN_B P61769 Beta-2-microglobulin X-ray 2.00 2021-01-22 94.06 0.98 0.02 ok
7CIQ_B P61769 Beta-2-microglobulin X-ray 1.59 2020-07-08 94.06 0.98 0.02 ok
7T11_B P62873 Guanine nucleotide-binding protein G(I)/G( EM 2.70 2021-11-30 97.06 0.98 0.01 ok
7VKT_C P62873 Guanine nucleotide-binding protein G(I)/G( EM 2.90 2021-10-01 97.06 0.98 0.01 ok
7T10_B P62873 Guanine nucleotide-binding protein G(I)/G( EM 2.50 2021-11-30 97.06 0.99 0.01 ok
7CIS_B P61769 Beta-2-microglobulin X-ray 2.10 2020-07-08 94.06 0.99 0.01 ok
7ENV_A P25440 Bromodomain-containing protein 2 X-ray 2.45 2021-04-19 64.06 0.98 0.01 ok
7Q90_A Q6IQ55 Tau-tubulin kinase 2 X-ray 1.60 2021-11-11 48.84 0.98 0.01 ok
7Q8Y_A Q6IQ55 Tau-tubulin kinase 2 X-ray 1.60 2021-11-11 48.84 0.98 0.01 ok
7Q8Z_A Q6IQ55 Tau-tubulin kinase 2 X-ray 1.57 2021-11-11 48.84 0.99 0.01 ok
7CIQ_A O78189 MHC class I antigen X-ray 1.59 2020-07-08 86.50 0.99 0.01 ok
7EBA_A P34913 Bifunctional epoxide hydrolase 2 X-ray 2.30 2021-03-09 93.31 0.99 0.00 ok
7CIS_A O78189 MHC class I antigen X-ray 2.10 2020-07-08 86.50 1.00 0.00 ok
7Q8V_A Q5TCY1 Tau-tubulin kinase 1 X-ray 2.13 2021-11-11 51.06 0.99 0.00 ok
7Q8W_A Q5TCY1 Tau-tubulin kinase 1 X-ray 2.02 2021-11-11 51.06 0.99 0.00 ok
7VDS_C Q15078 Cyclin-dependent kinase 5 activator 1 X-ray 3.05 2021-09-07 72.94 1.00 0.00 ok
7CIR_A O78189 MHC class I antigen X-ray 1.81 2020-07-08 86.50 1.00 0.00 ok
7VDQ_C Q15078 Cyclin-dependent kinase 5 activator 1, p25 X-ray 2.91 2021-09-07 72.94 1.00 0.00 ok
7DYN_A O78189 MHC class I antigen X-ray 2.00 2021-01-22 86.50 1.00 0.00 ok
7VDR_C Q15078 Cyclin-dependent kinase 5 activator 1, p25 X-ray 2.55 2021-09-07 72.94 1.00 0.00 ok
7VDP_C Q15078 Cyclin-dependent kinase 5 activator 1, p25 X-ray 2.09 2021-09-07 72.94 1.00 0.00 ok

Click a column header to sort. FRAUD = confidence-weighted error; a structure is flagged confidently wrong when mean pLDDT > 70 yet TM-score < 0.5.