the big data story that never was

What was the issue with NHS England's flagship data sharing initiative and what does its demise mean for Big Data?

I had mixed emotions when the Government decided to axe [1], a database of pseudo-anonymised NHS patient data to be made available for research by universities, drug companies and insurers. First, I felt vindicated: I had serious concerns about the way the programme was being rolled out and the ethical framework surrounding it. This quickly dissolved when I realised that the only reason I had any opinion at all on was because I had researched, planned and drafted an article on the subject for this publication that I now wouldn’t be able to write. I also realised that my opinions on the project were not as negative as I had first thought. Was really such a bad idea?

Big Data is a buzzword in Public Health research at the moment. It is not, however, a recent fad. In 1989, the NHS started to collect hospital patient data in Hospital Episode Statistics (HES), which was essential in drawing on trends to uncover the Stafford Hospital Scandal [2]. But the benefits of Big Data extend far beyond exposing underperforming hospitals: it enables diverse research, from uncovering drug side effects, to furthering our understanding of disease pathophysiology.

In 2013, under this mandate, the Health and Social Care Information Centre (HSCIC) proposed to extract GP records and thus create a central database of all NHS patient records for research purposes. However, never progressed very far, and the decision to scrap the project came after a report published in July 2016 by Dame Fiona Caldicott, the National Data Guardian, into data handling in the NHS [3].

By the end, the HSCIC database contained a lot of raw patient data, including NHS numbers, Post Codes, and medical records. These records contained potentially sensitive information, such as HIV status. Clearly there are fears that, from this pseudo-anonymised data, patient names could be re-identified.

Dame Caldicott’s report concluded that, while there was a clear commitment to secure handling of data, staff were challenged “in translating their commitment into reliable practice” [3]. Interestingly, public trust in the NHS’s ability to secure data is strong, and as long as so-called ‘data-handlers’ properly adhere to legislation such as the Data Protection Act, the risk to patient data is small. However, there was a call for “harsher sanctions for intentional or malicious breaches” [3].

Nevertheless, NHS England stated that program was abandoned as a direct result of this report. Perhaps they realised that the ethical framework required for to function was not in place.

I do not believe that sharing of patient data is part of some elaborate Orwellian conspiracy; there are clearly real benefits that could emerge from this age of Big Data. But it is important for us to question how such data is handled, and the ethical issues surrounding consent and anonymity of information.

For me, the more important issue surrounds the ethics of consent and the appropriate use of patient data. The guidelines on handling patient data set out in the 1997 Caldicott Report concluded it must be used in the ‘direct care’ of patients, defined as: “a clinical, social or public health activity concerned with the prevention, investigation and treatment of illness and the alleviation of suffering [of patients]” [4].

The principle of providing a direct clinical benefit still underpins the sharing of patient records with assumed consent. But there is certainly a large question mark over how potential benefits emerging from patient data can be justified as providing a ‘direct clinical benefit’ given that many of those whose information will be used will not benefit in any way.

While assumed consent may be reasonable and, more importantly, ethical, given the potential benefits of Big Data, it appears somewhat dishonest to sanction the sharing of data on such an arguably tenuous connection to the definition of ‘direct care’ set out in the Caldicott Report. Conversely, it seems questionable whether a report penned nearly two decades ago could have envisaged the way that data is being used today, let alone regulate it. New legislation is required to create a more appropriate and transparent means to determine the need for data sharing.

In the new report, Dame Caldicott acknowledged that sharing information is key to providing excellent care. In most instances, however, she states that anonymised data is sufficient, and that this may be given without the need for consent [4]. There are of course times when confidential data would be required for research or for monitoring care quality. It was therefore suggested that patients should have the right to opt-out of the sharing of confidential information.

There needs to be an extensive debate between health care professionals and the public to establish a rigorous framework so that data is ethically shared and processed. There cannot be different and seemingly conflicting rules for different circumstances. This current void will do little to ease the fears of people who are wary of the way that their personal data is distributed, and if large numbers of patients opt-out of such agreements, the power of Big Data is greatly reduced. We are witnessing a paradigm shift in the way that we use patient data. Surely it is appropriate to call for a new open and consistent system of ethical approval?

Rather than being exonerated by the scrapping of, I cannot help but feel that Dame Caldicott’s report, and the debate ought to follow, should have been conducted before the project was launched. The attempt to rush through such an ambitious project without due consideration of its ethical framework has led to its demise. could have been an exciting innovation used to make key advances in medicine. Instead, it has been cancelled after three years without realising any of its goals. Although the then Life Sciences Minister George Freeman maintained that there was still a commitment “to realising the benefits of sharing information” [1], I fear that a modernising endeavour on the scale of, may never again take shape.

Edward Arbe-Barnes is a third year medical student at Magdalen College.


Get in touch!

Contact us at

Oxford Medical School Gazette
Medical School Office

Academic Centre, Level 2

John Radcliffe Hospital

Oxford, UK


  • Facebook
  • Instagram

© Oxford Medical School Gazette 2019