A deal between Google’s DeepMind AI arm and Royal Free London NHS Foundation Trust has been heavily criticised over how it handled patient data.
As part of the deal, the Royal Free shared data of around 1.6 million patients with DeepMind. According to the BBC the records were to be used to build an app that would alert doctors about patients that were at risk of acute kidney injury.
The deal only became public knowledge in February 2016, around three months after the data sharing had occurred. Both parties were criticized over a lack of public consultation about the deal.
Now, an academic paper written by Julia Powles, a Cambridge University academic and Hal Hodson, a journalist with New Scientist, has criticised aspects of the arrangement, in particular the lack of public engagement. “The failure on both sides to engage in any conversation with patients and citizens is inexcusable,” the report said.
“We do not know––and have no power to find out––what Google and DeepMind are really doing with NHS patient data, nor the extent of Royal Free’s meaningful control over what Google and DeepMind are doing,” the report added. Additionally, “the amount of data transferred is far in excess of the requirements of those publicly stated needs.”
None of the millions of patients identified by the report’s authors were informed about the data sharing agreement with DeepMind, nor were they asked for their consent.
What’s particularly worrying from a public interest point of view is that the report found: “the data transfer was done without consulting relevant regulatory bodies, with only one superficial assessment of server security, combined with a post-hoc and inadequate privacy impact assessment”.
Any indications about what would happen with the data came from public relations statements, not independent oversight or legally-binding documents.
In a statement sent to Infosecurity, the ICO said: "Our investigation into the sharing of patient information between the Royal Free NHS Trust and Deep Mind is close to conclusion. We continue to work with the National Data Guardian and have been in regular contact with the Royal Free and Deep Mind who have provided information about the development of the Streams app. This has been subject to detailed review as part of our investigation."
DeepMind and the Royal Free issued a joint statement to Infosecurity Magazine, which claimed the report had misunderstood the nature of the agreement. “This paper completely misrepresents the reality of how the NHS uses technology to process data,” the statement said. “It makes a series of significant factual and analytical errors, assuming that this kind of data agreement is unprecedented.
“In fact, every trust in the country uses IT systems to help clinicians access current and historic information about patients, under the same legal and regulatory regime."
DeepMind added that the data is not shared with Google, and cannot leave England.
The report's authors dismissed Google DeepMind's statement. They told Infosecurity Magazine that, "public relations statements are one thing, but the agreement analysed in the article was signed by Google, and the simple fact is that there was no explicit prohibition on Google touching the data."
Powles and Hodson added that DeepMind getting into the healthcare sector is a "break from the norm. These companies are entirely different to specialised health IT and infrastructure providers, and the sweeping analogy does a disservice to the public. Scrutiny of this deal by public institutions is ongoing, and it is telling that the agreement under fire was voluntarily remade by the parties only halfway through its proposed two-year term."
This case raises the issue of how much sensitive patient data should be shared beyond the NHS, how much awareness there should be, and how much regulatory oversight there should be over third-party data sharing agreements.
Phil Booth, coordinator of medConfidential, told Infosecurity : “Accountability is vital - if patients were to be able to see which organizations had a copy of their medical history, those organizations would be held accountable for the data they hold, and the mechanisms by which they protect it.
“Every flow of data into, across and out of the NHS and care system should be consensual, safe, and transparent. With current technology that is entirely possible; and it is becoming a necessary standard by which an organization shows that it takes information security seriously.”