The Friends and Family Test - power to the patients?

In what the government is describing as an historic moment, the NHS has published its first set of data on how hospital patients rate their experience.

Since April, people in England have been asked how likely they are to recommend the hospital where they were treated. With only three months of data, the statistics will be of limited use. Of perhaps greater significance - at least at this stage - is how the survey is conducted.

Designed to shine a light on the best and worst of the NHS's hospitals, it's less full-beam spotlight and more 40W flashlight. It might yet become a more useful indicator, but it certainly shouldn't be regarded as a panacea for poor patient care.

Trip Advisor for the NHS?

That's how Tim Kelsey, the National Director for Patients and Information, described the Family and Friends Test, arguing that the NHS needed to learn the lessons of "transparency and feedback".

Hospitals now pose the following question to patients:

"How likely are you to recommend our ward/A&E department to friends and family if they needed similar care or treatment?"

So far, it's only patients on acute wards and those who are treated in A&E who are asked to grade their experience on the six point scale (where possible answers range from "extremely likely" to "extremely unlikely").

Once the proportion of positive and negative responses has been calculated, a hospital is accorded a score. The higher the score (on a scale of -100 to 100), the more satisfied the patients. For the last three months, the average score on an inpatient ward fluctuated around 70; for A&E it was about 53. (If you want to see how your local hospital performed, you can go to the NHS Choices website.)

Ratings you can trust?

In the last round, the average response rate at the 157 trusts that provide acute care was close to 25%. For the 144 trusts with A&E departments, the response rate was much lower - under 8%. At Warwick Hospital's A&E department, only three people were surveyed during the whole of June.

With a low response rate, a hospital is less able to make helpful improvements to the way it deals with patients. On the other hand, it might have collected a small sample of positive responses that don't speak of the poor care that patients are more likely to experience.

There's every reason to expect that hospitals will extend the survey's reach. This much is suggested by the fact that response rates improved month on month between April and June. 

Potentially more problematic is the fact that hospitals are free to conduct the survey however they see fit. Although the phrasing of the question is standardised and it's multiple choice, there's no definitive methodology for collecting the data. In other words, NHS trusts can potentially pick and choose the patients they survey. 

This means we're more likely to see what's called a 'mode effect', where different methods of administering a survey lead to variations in the data that's collected. Some hospitals will send an SMS to patients two days after they've been discharged; others will ask patients to fill out forms during their stay.

In each case a hospital might not reach out to a representative sample of patients - for instance, someone who's elderly might be less likely to own a mobile phone. This has led one patient group to suggest that the survey is "open to gaming" - that certain patients might be encouraged to respond by staff as long as there's a financial incentive in posting a positive score.

There's also another, more fundamental concern about the survey. While the question might appear straightforward, the Institute for Government (IfG) has explained that its phrasing has been the subject of some debate: in its trial phase, "many people objected to the very idea of recommending something like a trip to hospital".

Better, worse, or no improvement?

The Friends and Family Test has the potential to be a helpful measure. 

While the test is currently limited to certain hospital departments, the government intends that every NHS service will be using the survey by April 2015. And although there are other ways of measuring patient satisfaction, such as the Department of Health's statistics from the annual Patient Experience Survey, the Friends and Family Test provides a regular set of data. NHS England says that most hospitals will review their feedback weekly, which means that they'll be in a better position to make 'real-time' improvements. NHS trusts are also "expected" to ask more detailed follow-up questions, although - once again - it's ultimately up to local hospitals if and how they do this.

The survey is not a cure-all. The government has suggested that it will help to prevent another crisis like that which occurred at Mid Staffordshire NHS Trust. While patient feedback might help to make hospitals better in many small ways, it's unlikely that egregious failures of care will show up in these statistics. For this, we have other (imperfect) measures. 

The government sees the Friends and Family Test as a vital intervention. But even if the NHS is already a marketplace, it's not clear that patients will truly become empowered 'consumers'.