Last week saw a dramatic development in the Government's handling of the NHS's care.data project aimed at linking GP records to hospital records. Less than a month before starting to extract copies of everyone's primary care records into a central database, NHS England has "paused" the project for six months so that they "can allow more time to build understanding of the benefits of using the information, what safeguards are in place, and how people can opt out". This follows increasingly vocal opposition from the medical community (including the Royal College of GPs, the General Pharmaceutical Council and the British Medical Association), patient groups, the Information Commissioner's Office (ICO), and an increasingly vociferous media campaign. Further fuel has been added to the flames by a number of GPs who have publicly declared their intentions to opt out and to advise their patients to opt out themselves.

Most of the worry is about how potentially sensitive patient information will be used, who will have access to it (and for what reasons), as well as its security. Such fears are not just hypothetical, given that examples of lost patient notes and what appear to be the misuse of sensitive patient information (even for the best of intentions) already exist. This has led many commentators to speculate whether the initiative might be permanently holed below the waterline by people opting out and, indeed, whether care.data might end up in the same political graveyard as the poll tax and the ID Cards scheme? With opt outs currently running at 5 per cent and rising, perhaps these concerns are well founded.

Understandably, confusion abounds and it's hard for people to know how to opt out because the leaflet telling everyone was sent as unaddressed (junk) mail to every household (except for those who have opted out of receiving junk mail, who got nothing) and in any case the leaflet wasn't clear on how to opt out. Confused? So is everyone else.

The ICO has consistently said that citizens should be asked for their informed consent before their personal records are used for purposes other than those for which they primarily exist. In other words they think the law requires an opt-in mechanism. There is quite a gap between this regulator's position and the approach taken by NHS England the next six months will therefore be particularly challenging but also critical to the future delivery of healthcare.

It's also easy to see why the general public is becoming increasingly concerned. Conversations with GPs are amongst the most private and personal of all conversations outside your family and friends, indeed sometimes more private even than those with close family. Pseudonymisation, in which all personal identifiers are replaced with meaningless codes, can be done to an existing NHS standard but there is an unresolved debate over whether it is feasible to do it at source before the information leaves the GP system. In addition, the more that the "open" data concept publishes and links large data sets, as is happening at an increasing rate across society generally, the greater the likelihood that a person can be re-identified by multiple other means. This suggests that any expectations that privacy can be maintained in the future may become unachievable and NHS England could find itself offering privacy just at the point where such privacy can no longer realistically be achieved. Things may well look very different in five years' time, as no government can irrevocably commit all future governments to a particular course of action.

And yet this data is massively valuable clinically. As a nation we have very little idea of clinical cause and effect in primary care, whereas hospital episode statistics (HES) have been telling us for decades what works and what doesn't in secondary care. Indeed, HES has provided significant benefits in terms of improving patient outcomes through analysis of hospital activities and performance. Moreover, there have been no recorded examples of data loss leading to patient or public harm, or the actual (as opposed to theoretical) identification of individual patients. There is game-changing potential in the use of this linked data, possibly more than enough to move the needle on NHS affordability. So (not for the first time) the interests of the population and the interests of the individual are tugging in opposite directions.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.