a blog of books every doctor should read

Big Data

The capacity of modern technology to produce enormous amounts of information is growing. This book explores how this is transforming all facets of society, from healthcare to astronomy, from financial markets to social media. The authors see Big Data as ‘…a vital economic input’ (page 5) with the potential to ‘…produce useful insights or goods and services of significant value’ (page 2). They predict that ‘…in the future, our understanding will be driven more by abundance of data rather than by hypotheses’ (page 70). The book is furnished with several interesting anecdotes which illustrate the application of Big Data for example in detecting financial, sporting and academic fraud (pages 27 and 28).

The thrust of the book’s argument is that Big Data produces strong correlations that could be applied to make accurate predictions (page 55). The book illustrates this with healthcare-related examples such as the prediction of epidemics, health risks and health behaviours (pages 57 and 94). They also show how the application of Big Data may forecast patient deterioration, and hospital infection and readmission rates (page 60). The authors illustrate the relative advantage of large data with examples such as the design of targeted treatments using whole genome sequencing DNA analysis (page 25). They discuss the crucial impact of Big Data on research arguing that it would dispense with the arduous and biased process of population sampling they label ‘… an artefact of a period of information scarcity…’ (page 13).

A key and perhaps controversial assertion made by the authors is that Big Data provides sufficient correlative information to make the search for cause unnecessary. They argue that both correlation and cause are rarely completely proven and that ‘…we don’t always need to know the cause of a phenomenon: rather, we can let the data speak for itself’ (page 14). They admit the impreciseness or messiness of correlation but nevertheless argue that it generates sufficient information to highlight relevant trends and associations (page 14).

Big Data is not without its risks and consequences and the book mentions poor data quality, biased data application, ‘the dictatorship of data‘ (page 163), and ‘predictive policing‘ (page 158). Data may also ‘fail to capture what it purports to quantify’ (page 166). The authors review the implications of Big Data on data security and advocate a new privacy framework, ‘…focused less on individual consent at the time of collection and more on holding data users accountable for what they do’ (page 173). They illustrate data misuse with a story of Robert McNamara who, as US Secretary of Defence during the Vietnam War, allowed his obsessive reliance on data to cloud his judgment (page 163).

A worrying implication of the rise of Big Data is what the authors call ‘the demise of the expert‘ (page 139). This is the expected outcome of ‘data-driven decisions’ which the authors predict will increasingly ‘…augment or overrule human judgment’ (page 141). They point out that the influence of experts in many fields is waning because ‘…the skills necessary to succeed in the workplace are changing’ (page 141). They said ‘those who can analyze big data may see past the superstitious and conventional thinking not because they are smarter, but because they have the data’ (page 143).

The information age has made Big Data inevitable and this book addresses the consequences of this on society. The authors predict the widespread applications of large amounts of information particularly to healthcare. Their outlook on correlation, and its relative importance over causation, has significant implications on medical research and the understanding of risk factors. The book’s prediction of the ‘demise of the expert’ is a clarion call to physicians to reassess how they engage with data and to justify their future relevance. Uniquely, the book dispenses with an introduction and preface; this is refreshing as these are often uninformative sections of books.

OVERALL ASSESSMENT

Healthcare produces and uses large amounts of data and this activity is increasing. Data is used for clinical practice, healthcare planning and health research. Doctors therefore need to be aware of how the increasing applications of Big Data would influence their practice. This book provides important insights on these issues and I highly recommend it.