News

CASP16 Call for Targets

04/08 wwPDB News

CASP (Critical Assessment of protein Structure Prediction) is in search for targets.CASP (Critical Assessment of protein Structure Prediction) is in search for targets.

CASP (Critical Assessment of protein Structure Prediction) experiments are held every two years. Recent rounds have seen dramatic increases in modeling accuracy, resulting from the introduction of deep learning methods: In 2018, for the first time, the folds of most proteins were correctly computed [1]; in 2020, the accuracy of many computed protein structures rivaled that of the corresponding experimental ones [2]; in 2022, there was an enormous increase in the accuracy of protein complexes [3].

We have seen the beginning of what deep learning methods may achieve in structural biology. In addition to further increases in the accuracy of protein complexes, methods are being developed for RNA structures, organic ligand-protein complexes, and for moving beyond single macromolecular structures to compute conformational ensembles. Accurate computational methods together with experimental data also offer the prospect of probing previously inaccessible biological systems. CASP has expanded its scope to provide critical assessment in all these areas.

CASP is only possible with the generous participation of the experimental structural biology community in providing suitable targets: A total of over 1100 targets have been obtained over the previous CASP rounds. We are now requesting targets for the 2024 CASP16 experiment. We need challenge targets in the following areas:

Single protein structures: The 2020 and 2022 CASPs showed that, so far, Alphafold2 and methods built around it are by far the most accurate [4]. But there are limitations, particularly for some proteins where only a shallow sequence alignment is available and for very large proteins (more than 1000 amino acids). The best results also require substantial amounts of computing resources, well beyond that of the AlphaFold2 default settings. Many new methods are continuing to appear and these may remove some of the remaining difficulties. All types of protein targets are needed, but especially those with shallow sequence alignments, without structural templates, and large proteins.

Protein complexes: In the 2022 CASP15, advanced deep learning methods were applied to protein complexes for the first time [5]. The result was a huge improvement in accuracy compared with classical docking approaches. But overall, the results are still not at the level achieved for single proteins. So, in CASP16 we need all sorts of targets in this area so as to determine progress since then. We particularly need complexes where there is no evolutionary information across the protein-protein interfaces, for example, antibody-antigen complexes. (This CASP category is conducted in close collaboration with our colleagues at CAPRI - Critical Assessment of protein interactions [6]).

Nucleic acid structures and complexes: In recognition of the major role nucleic acid structures and complexes play in biology, CASP now includes this class of target. A number of papers claiming successful RNA structure computation using deep learning methods have been published, but those participating in the 2022 CASP RNA category performed less well than classical approaches, and no methods were able to effectively address the two RNA protein-complexes included [7]. CASP needs a wide variety of RNA, DNA, and complexes as targets to see if this situation has changed. (This CASP category is conducted in close collaboration with RNApuzzles [8]).

Organic ligand-protein complexes: This area is of major importance for computer-aided drug discovery. Earlier, there have been community experiments to assess the accuracy of methods, particularly SAMPL, CSAR, D3R, and a new one, CACHE, has recently started. These challenges have drawn strong international participation from researchers in both academia and industry. Here too, a number of promising deep learning papers have appeared, but in the 2022 CASP15 pilot, classical methods were still superior [9]. So, we need appropriate targets to see if progress has been made since. Ideally, these should be sets of three-dimensional protein-ligand complexes from drug discovery projects, but single targets would also be appreciated. Additionally, where available, we will assess non-structural quantities such as affinities or affinity rankings and other properties of pharmaceutical interest when these are available (small molecule pKs, and DMPK related properties).

Ensembles of macromolecule conformations: It is now widely recognized that proteins and nucleic acids often adopt multiple conformations that can underpin their functions. In these cases, considering only a single protein or RNA conformation may be a significant oversimplification. The 2022 CASP15 included a pilot experiment to assess methods for computing multiple conformations, with encouraging results [10], but with limitations imposed by the available experimental data. For 2024, we seek not only cases of multiple experimental three-dimensional structures for the same macromolecule but also other types of data that might be used for assessment of computed conformation ensembles such as cryoEM, NMR, X-ray crystallography, SAXS, and/or cross-link data.

Integrative modeling: The more powerful computational methods open up new possibilities for combination with sparse or low-resolution experimental data to investigate previously inaccessible biological structures and machines. CASP is interested in exploring these possibilities and so requests experimentally difficult targets where structure has nevertheless been obtained. In appropriate cases, we expect to be able to collaborate with other experimental groups to provide appropriate data from NMR, cross-linking or SAXS.

There are three avenues to contribute a target to CASP:

  1. (preferred) Submit directly to CASP through our web-interface (requires a quick registration if you do not have an account with us).
  2. Email to [email protected] with your target suggestions or to discuss any questions.
  3. Submit your structure to the PDB (on-hold) and designate it as a CASP target through PDB’s submission interface.

The timeline for the 2024 CASP requires that targets are submitted starting now and until July 1. We would like to hear from you as soon as possible if you may have something suitable or have suggestions about other target sources. In order to maintain rigor, the experimental data for a target must not be publicly available until after computed structures have been collected. For assessment, CASP requires the experimental data by August 15, but the data can remain confidential after that. Target providers are invited to contribute to papers [11-15] for a special CASP issue of the journal Proteins.

CASP organizers: John Moult, Krzysztof Fidelis, Andriy Kryshtafovych, Torsten Schwede, Maya Topf

  1. Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical assessment of methods of protein structure prediction (CASP)-Round XIII. Proteins 2019;87(12):1011-1020.
  2. Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical assessment of methods of protein structure prediction (CASP)-Round XIV. Proteins 2021;89(12):1607-1617.
  3. Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical assessment of methods of protein structure prediction (CASP)-Round XV. Proteins 2023;91(12):1539-1549.
  4. Simpkin AJ, Mesdaghi S, Sanchez Rodriguez F, Elliott L, Murphy DL, Kryshtafovych A, Keegan RM, Rigden DJ. Tertiary structure assessment at CASP15. Proteins 2023;91(12):1616-1635.
  5. Ozden B, Kryshtafovych A, Karaca E. The impact of AI-based modeling on the accuracy of protein assembly prediction: Insights from CASP15. Proteins 2023;91(12):1636-1657.
  6. Lensink MF, Brysbaert G, Raouraoua N, Bates PA, Giulini M, Honorato RV, van Noort C, Teixeira JMC, Bonvin A, Kong R, Shi H, Lu X, Chang S, Liu J, Guo Z, Chen X, Morehead A, Roy RS, Wu T, Giri N, Quadir F, Chen C, Cheng J, Del Carpio CA, Ichiishi E, Rodriguez-Lumbreras LA, Fernandez-Recio J, Harmalkar A, Chu LS, Canner S, Smanta R, Gray JJ, Li H, Lin P, He J, Tao H, Huang SY, Roel-Touris J, Jimenez-Garcia B, Christoffer CW, Jain AJ, Kagaya Y, Kannan H, Nakamura T, Terashi G, Verburgt JC, Zhang Y, Zhang Z, Fujuta H, Sekijima M, Kihara D, Khan O, Kotelnikov S, Ghani U, Padhorny D, Beglov D, Vajda S, Kozakov D, Negi SS, Ricciardelli T, Barradas-Bautista D, Cao Z, Chawla M, Cavallo L, Oliva R, Yin R, Cheung M, Guest JD, Lee J, Pierce BG, Shor B, Cohen T, Halfon M, Schneidman-Duhovny D, Zhu S, Yin R, Sun Y, Shen Y, Maszota-Zieleniak M, Bojarski KK, Lubecka EA, Marcisz M, Danielsson A, Dziadek L, Gaardlos M, Gieldon A, Liwo A, Samsonov SA, Slusarz R, Zieba K, Sieradzan AK, Czaplewski C, Kobayashi S, Miyakawa Y, Kiyota Y, Takeda-Shitaka M, Olechnovic K, Valancauskas L, Dapkunas J, Venclovas C, Wallner B, Yang L, Hou C, He X, Guo S, Jiang S, Ma X, Duan R, Qui L, Xu X, Zou X, Velankar S, Wodak SJ. Impact of AlphaFold on structure prediction of protein complexes: The CASP15-CAPRI experiment. Proteins 2023;91(12):1658-1683.
  7. Das R, Kretsch RC, Simpkin AJ, Mulvaney T, Pham P, Rangan R, Bu F, Keegan RM, Topf M, Rigden DJ, Miao Z, Westhof E. Assessment of three-dimensional RNA structure prediction in CASP15. Proteins 2023;91(12):1747-1770.
  8. Magnus M, Antczak M, Zok T, Wiedemann J, Lukasiak P, Cao Y, Bujnicki JM, Westhof E, Szachniuk M, Miao Z. RNA-Puzzles toolkit: a computational resource of RNA 3D structure benchmark datasets, structure manipulation, and evaluation tools. Nucleic Acids Res 2020;48(2):576-588.
  9. Robin X, Studer G, Durairaj J, Eberhardt J, Schwede T, Walters WP. Assessment of protein-ligand complexes in CASP15. Proteins 2023;91(12):1811-1821.
  10. Kryshtafovych A, Montelione GT, Rigden DJ, Mesdaghi S, Karaca E, Moult J. Breaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15. Proteins 2023;91(12):1903-1911.
  11. Kretsch RC, Andersen ES, Bujnicki JM, Chiu W, Das R, Luo B, Masquida B, McRae EKS, Schroeder GM, Su Z, Wedekind JE, Xu L, Zhang K, Zheludev IN, Moult J, Kryshtafovych A. RNA target highlights in CASP15: Evaluation of predicted models by structure providers. Proteins 2023;91(12):1600-1615.
  12. Alexander LT, Durairaj J, Kryshtafovych A, Abriata LA, Bayo Y, Bhabha G, Breyton C, Caulton SG, Chen J, Degroux S, Ekiert DC, Erlandsen BS, Freddolino PL, Gilzer D, Greening C, Grimes JM, Grinter R, Gurusaran M, Hartmann MD, Hitchman CJ, Keown JR, Kropp A, Kursula P, Lovering AL, Lemaitre B, Lia A, Liu S, Logotheti M, Lu S, Markusson S, Miller MD, Minasov G, Niemann HH, Opazo F, Phillips GN, Jr., Davies OR, Rommelaere S, Rosas-Lemus M, Roversi P, Satchell K, Smith N, Wilson MA, Wu KL, Xia X, Xiao H, Zhang W, Zhou ZH, Fidelis K, Topf M, Moult J, Schwede T. Protein target highlights in CASP15: Analysis of models by structure providers. Proteins 2023;91(12):1571-1599.
  13. Alexander LT, Lepore R, Kryshtafovych A, Adamopoulos A, Alahuhta M, Arvin AM, Bomble YJ, Bottcher B, Breyton C, Chiarini V, Chinnam NB, Chiu W, Fidelis K, Grinter R, Gupta GD, Hartmann MD, Hayes CS, Heidebrecht T, Ilari A, Joachimiak A, Kim Y, Linares R, Lovering AL, Lunin VV, Lupas AN, Makbul C, Michalska K, Moult J, Mukherjee PK, Nutt WS, Oliver SL, Perrakis A, Stols L, Tainer JA, Topf M, Tsutakawa SE, Valdivia-Delgado M, Schwede T. Target highlights in CASP14: Analysis of models by structure providers. Proteins 2021;89(12):1647-1672.
  14. Lepore R, Kryshtafovych A, Alahuhta M, Veraszto HA, Bomble YJ, Bufton JC, Bullock AN, Caba C, Cao H, Davies OR, Desfosses A, Dunne M, Fidelis K, Goulding CW, Gurusaran M, Gutsche I, Harding CJ, Hartmann MD, Hayes CS, Joachimiak A, Leiman PG, Loppnau P, Lovering AL, Lunin VV, Michalska K, Mir-Sanchis I, Mitra AK, Moult J, Phillips GN, Jr., Pinkas DM, Rice PA, Tong Y, Topf M, Walton JD, Schwede T. Target highlights in CASP13: Experimental target structures through the eyes of their authors. Proteins 2019;87(12):1037-1057.
  15. Kryshtafovych A, Albrecht R, Basle A, Bule P, Caputo AT, Carvalho AL, Chao KL, Diskin R, Fidelis K, Fontes C, Fredslund F, Gilbert HJ, Goulding CW, Hartmann MD, Hayes CS, Herzberg O, Hill JC, Joachimiak A, Kohring GW, Koning RI, Lo Leggio L, Mangiagalli M, Michalska K, Moult J, Najmudin S, Nardini M, Nardone V, Ndeh D, Nguyen TH, Pintacuda G, Postel S, van Raaij MJ, Roversi P, Shimon A, Singh AK, Sundberg EJ, Tars K, Zitzmann N, Schwede T. Target highlights from the first post-PSI CASP experiment (CASP12, May-August 2016). Proteins 2018;86 Suppl 1(Suppl 1):27-50.

News Index