The Challenge of Saving Lives with Big Data

Big data is revolutionising many fields, including medicine. However, the complexity of medical data presents a particular challenge, with many properties that are hard to quantify. The nature of patient data may range from molecular and genetic information, to demographics and social interaction.

Analysis of massive health datasets to find hidden relationships has huge potential. Big Data can be used to help identify new drug targets, or accurately predict the progression and occurrence of diseases, thus allowing earlier clinical interventions. Progress in machine learning has been instrumental to improving these predictive models. An artificial neural network can be trained to a large dataset for a given disease or complication. The resulting model can then be used to predict disease in a new dataset.

Currently, several such predictive models have been developed, and have been shown to be effective in clinical trials. However, there is still a lack of evidence that this will translate to improved quality of care. The benefits of early prediction are limited in the case of diseases that lack effective treatment. Additionally, there is some concern that when deployed on a larger scale, predictive models using Big Data will lead to over-diagnosis and over-treatment. These are challenges that will need to be overcome before Big Data sees more widespread medical use.

Big data in medicine may provide the opportunity to view human health holistically, through a variety of lenses, each presenting an opportunity to study different scientific questions. Here we characterized health data by several axes that represent different properties of the data. The potential scientific value of collecting large amounts of health data on human cohorts has recently been recognized, with a rapid rise in the creation of large-scale cohorts aiming to maximize these axes. However, since maximizing each axis requires both resources and effort, it is inevitable that some axes come at the expense of others. Analysis of big data in health has many challenges and is in some sense a double-edged sword. On one hand, it provides a much wider perspective on states of health and disease, but on the other hand, it provides the temptation to delve into the details of molecular descriptions that may miss the big picture (as in the ‘seeing the whole elephant’ analogy). In addition, real-world evidence that it will translate into improved quality of care is currently lacking. However, the potential to improve healthcare is still immense, especially as patients’ conditions and medical technologies become more and more complex over time. With the collection of more deeply phenotyped large-scale data, many scientific questions about disease pathogenesis, classification, diagnosis, prevention, treatment and prognosis can be studied and can potentially lead to new discoveries that may eventually revolutionize medical practice.

Shilo, S., Rossman, H., & Segal, E. (2020). Axes of a revolution: challenges and promises of big data in healthcare. Nature Medicine, 26(1), 29-38. doi: 10.1038/s41591-019-0727-5