Research Category: Challenges and Benchmarking

Recent advances in mobile health have demonstrated great potential to leverage sensor-based technologies for remote monitoring of health and disease – particularly for diseases affecting motor function such as Parkinson’s disease (PD). However, converting sensor-based data streams into digital biomarkers is complex and no methodological standards have yet evolved to guide this process.

The Parkinson’s Disease Digital Biomarker (PDDB) DREAM Challenge was first-of-its-kind, designed to benchmark methods for processing sensor data for development of digital signatures reflective of PD. Over 440 data experts from six continents participated in the PDDB DREAM Challenge.

The challenge was divided into two sub-challenges:

• In the first sub-challenge, participants used accelerometry and gyroscope data from mPower – a large mobile health study where over 15,000 individuals with PD or controls used their iPhones to, among other things, perform short walk and balance tests – to extract features that were used to predict whether the user had PD.
• In the second sub-challenge, participants extracted features for three different Parkinson’s symptoms from a study funded by The Michael J. Fox Foundation where patients performed tasks while wearing three wearable sensors. These features were used to predict clinician-assessed disease severity for tremor, dyskinesia and bradykinesia.
  

Cancer is a disease of the genome: sequential accumulation of DNA mutations lead to both direct and indirect changes in the structure and abundance of the proteins that perform most of the functional activities within the cell. While methodologies for the analysis of cancer genomes and transcriptomes have undergone rapid benchmarking and standardization, our understanding of how best to analyze the cancer proteome remains less-developed. In particular, there are key questions remaining in how to infer the abundances of peptides not detected in a subset of samples, in optimizing database searches to detect cancer-specific peptides caused by point-mutations, alternative transcript isoforms or fusion genes, in understanding the association between DNA, mRNA and protein data, and in performing robust absolute quantitation.

To address these issues, we have set up the NCI-CPTAC DREAM Challenge as a community-based collaborative competition of researchers from across the world working together to answer key questions in cancer proteomics, focused around the integration of diverse data-types. The CPTAC effort includes the collection of  tumour/normal pairs with matched genomic, transcriptomic and proteomic characterization, which provide a unique data to run the proteogenomic challenge.

Multiple myeloma (MM) is a cancer of the plasma cells in the bone marrow, and its clinical course depends on a complex interplay of clinical traits and molecular characteristics of the plasma cells. Since risk-adapted therapy is becoming standard of care, there is an urgent need for a precise risk stratification model to assist in therapeutic decision-making. While progress has been made, there remains a significant opportunity to improve patient stratification to optimize treatment and to develop new therapies for high-risk patients. The MM DREAM Challenge aims to accelerate the development and evaluation of such risk models in multiple myeloma.

The ICGC-TCGA DREAM Somatic Mutation Calling – RNA (SMC-RNA) Challenge is a community-based collaborative competition of researchers from across the world charged to rigorously assess the accuracy of methods used in cancer-associated RNA-seq analysis, with focus in two areas: quantification of known isoforms and detection of novel fusion transcripts. The SMC-RNA Challenge will analyze dozens of samples created to have known alterations representing different tumor types, allowing confidence that the winning methods will be generalizable across the broad range of human cancers.