WHY BETTER MEASURES LEAD TO BETTER LIVES

In people centred services such as health and social care, our purpose is to help people to live well. We are not measuring widgets. We are measuring how well someone’s life has improved. We should care deeply about this. Choosing the wrong measures can put vulnerable people’s lives at risk. Choosing the right measures gives us an opportunity to help people to live well.

The go-to measure in most people centred services is targets: to complete an assessment within a set number of days, to spend a certain amount of time with a patient or to ‘fix’ X number of people. Managers set targets with good intentions. They believe that setting targets will improve outcomes for citizens, motivate staff and protect scarce resources. But do good intentions lead to good results?

Setting targets for individuals assumes that:

• People have a large degree of control over their work

• Achieving targets means better outcomes for citizens

In reality:

• Targets are always arbitrary, and never a reliable measure of performance

• Targets do not improve outcomes for citizens

• The system in which people work actually has the biggest impact on performance

When a manager sets targets and people can’t meet them easily, they either cheat the numbers or cheat the system in an effort not to be paid attention to.

The result is the very opposite of the good intention. The focus switches from improving things for the citizen to improving things for yourself. Instead of doing good, staff concentrate on looking good. This behaviour is rational and predictable.

The consequences of setting targets in people centred services could hardly be more serious; people are not properly understood, are misdiagnosed and their lives sometimes put at risk. For example, the seven-day target from contact to assessment used in social services means that the assessment may be done too quickly, missing important relevant information. A GP under pressure to see each patient in less than 10 minutes can misunderstand the true need and at worse, misdiagnose. The hospital target for ‘average length of stay’ can lead to patients being discharged prematurely, or without the right support networks in place.

Does it matter to the individual – or the organisation?

What is interesting is that these measures and many others like them, focus on what matters to the organisation, not what matters to the citizen. They relate to what has become the organisation’s de facto purpose – ‘meet government targets’, ‘achieve foundation status’ or ‘make a profit’, for example. They do not relate to the purpose from the citizen’s point of view, which is very different: ‘help me live my life well’ or ‘make me better’.

Organisation-centred measures are not used to foster learning or to understand the current system. Many of them are so arbitrary and out of context that they mean nothing.

If you spend time on an A&E ward, you may have seen the ‘performance boards’. Ask the staff if they ever look at them. Ask them if they ever use these targets to change the way they work. You will not be surprised to learn that they do not pay attention to them because they are so out of context and unconnected to the reality of the work. Even more informative is to ask the leaders of the service if they use the measures in their routine daily work. Again, you will probably discover that the only time they pay attention to the measures is either when they are in breach of the target, in crisis management mode, or at monthly management meetings, held in a room remote from the place where the work actually happens.

The test of a good measure

So what does a good measure look like?  The Vanguard Method only uses measures that pass the test of a good measure. Good measures:

• Relate to purpose from the citizen’s point of viewShow variation over time

• Help people to learn, understand and improve the system

• Are in the hands of the people doing the work

• Are used by leaders to take effective action on the system

In our experience, very few of the measures currently used in people centred services pass this test.

Good measures in people centred services can be split into ‘individual measures’ and ‘system measures’. Individual measures help to show whether an organisation’s service is actually helping people – the human beings, citizens and persons, with names, who needs help. System measures help to identify and remove obstacles that prevent delivery of that help. We call these ‘system conditions’.

Individual measures in people centred services relate to ‘what matters to the individual’ or ‘what a good life looks like for the individual’.

Clearly these are qualitative measures. Qualitative measures rely primarily on words as the unit of analysis and means of understanding. However, you can also use voice tone, loudness, cries, sighs, laughs, and many other forms of human communication. Many services that have applied the Vanguard Method use qualitative measures to show them how well they are helping individuals to achieve their purpose.

Understanding an individual’s purpose is achieved via sustained conversations that are reflected in spidergrams. The individual identifies what matters to them – for example ‘I’d like to be able to cook meals for myself’ – and rates on a scale of 1 to 10 how near they are towards achieving that. The organisation then works with the person to understand and achieve the goal.

This is an example of a spidergram; a qualitative leading measure that demonstrates (over time) how well a system is helping an individual to achieve their purpose.

Individual measures are called leading measures. Leading measures are the ones we use to understand and improve the system on a day-to-day basis, live in the work. They link directly to what matters to the citizen.

System-level measures

It is also important to have good measures at system level that help leaders to identify and eliminate the system-wide conditions that constrain the system’s ability to help a variety of different people, or in Vanguard terms, to ‘absorb variety’.

Such measures include:

• Volume of demands in

• Volume of demands out

• Number of repeat demands for the same case or issue

• End-to-end time either across the whole system or for parts of the system involving a specific process

There are some system-wide measures specific to certain people centered service:

• Number of care packages put in place

• Cost of care packages put in place

• Number of pieces of equipment issued

• Cost of equipment issued

The measures above help to identify what is happening in the system and importantly why it is happening, so that the system conditions can be addressed and eliminated. These measures are known as lagging measures because they apply after the event – after the real work with the citizen has been done.

There may be certain things common to many citizens that stop them from living the life that they choose. For example, someone might be lonely and want company. Loneliness may ‘present’ itself to the system in many different ways. However, the underlying cause of loneliness is likely to be a lack of family relationships, community support, or friendship networks. If loneliness is a common cause of people not living well, workers can do something on a locality level to address this, such as set up groups, connect people to each other, or improve community transport. In this case, a system-level measure might relate to community networks. The link between the two is that the system measure is derived from the individual measure.

The measurement challenge

The challenge for leaders and indeed regulators is that qualitative measures are not the norm and are often not readily available. They can’t be collected in a spreadsheet or written up on a side of A4 for the management board. The only way leaders can get meaningful view is to leave their office and go face-to-face with individuals to understand at firsthand what matters to them and if and how the system is helping them to achieve it. Some would argue that qualitative measures are less statistically and scientifically sound than quantitative ones. But people-centered services are not about products or machines. The aim is to measure how well someone’s life has improved, in their language and on their terms.

This is not to say that quantitative data is not needed, but it does require a different approach to measuring and also a different approach to inspection and regulation. Taking a different view of measurement in people centred services would lead to a significant improvement in results from the citizen’s perspective. At the heart of measurement would be the citizen and what matters to them. This perspective results in improved quality of life for citizens, fewer complaints, more satisfied staff and much less cost in the system.

We have learned what not to measure. Never measure the achievement of targets. Targets are always arbitrary, the data obtained from them is unreliable, and it cannot be used to improve. As W. Edwards Deming put it: ‘What do “targets” accomplish? Nothing. Wrong: their accomplishment is negative’.

Instead, we should measure whether we are helping individuals, on their own terms. The best people to do this are frontline staff and professionals, not managers. System-level measures tell us whether we are succeeding in removing system conditions. Leaders and managers use system-level measures to understand how well the system is performing overall.

But the most important lesson of all is that measurement in services dealing with people at vulnerable moments in their lives really matters. Get it wrong and the damage may be permanent, with huge implications for cost as well as wellbeing. Get it right and the consequences for individuals, their families and community can be profound.

---

Read similar articles in Edition Two of The Vanguard Periodical: The Vanguard Method in Financial Services. Download it here