Supporting structure prediction method development with Continuous Automated Model EvaluatiOn (CAMEO)
Juergen Haas, Xavier Robin, Anna Smolinski, Rafal Gumienny, Flavio Ackermann, Torsten F. Schwede · 2019
Protein structure prediction has become widely used in the life sciences as methods have matured significantly over the past 10 years. Today, most structure prediction workflows are fully automated. Consequently, establishing an automatized assessment and benchmarking process is key to sustained high-paced development of emerging methods. Continuously assessing structure prediction servers e.g. allows scientists to leverage the accumulated data to retrospectively select the best tool for a given scientific question. The Continuous Automated Model EvaluatiOn (CAMEO) platform is hinged on the weekly PDB release cycle allowing for a blind selection of prediction targets four days ahead of the release of the new deposition coordinates. It has been assessing predictions for over 6’700 targets in the 3D protein structure prediction category over 377 weeks, with currently about 20 new targets being assessed each week. CAMEO features baseline structure predictors in each of its categories. Additionally, we have recently implemented a “Best Single Template” baseline comparison resembling an upper “optimal alignment” limit by employing structural superposition to scan the ProteinDataBank (PDB) for the best available template at the time of target submission. This method helps identifying potential room for improvement of the template selection and alignment steps in automated protein structure modeling pipelines. Another integral part to CAMEO is the target selection, where we present the latest efforts on fully automated target validation