THE CASE AGAINST ISO CONTINUES

Written and prepared for FiLM 2016

‘Quality’ is set to sub-optimise Pathology Services: It’s the law!

Pathology services are obliged to be assessed (inspected) for conformance to ISO 15189, a ‘Quality Standard’. It was a key recommendation in the Barnes review and is in line with UK Government policy. The Department for Business, Innovation and Skills and the Department of Health agreed policy on accreditation sets out the following:

" UKAS (United Kingdom Accreditation Service) is the sole government recognised NAB (National Accreditation Body) in accordance with EU Regulation 765/2008 (the legislation that established the legal framework for accreditation in Europe).

UKAS’s role as the NAB thus applies in the health and care sectors as in all others. In accordance with EU and UK legislation health and social care quality assurance schemes should be developed with the engagement of UKAS.

Where new quality assurance schemes are planned in the NHS or social care, particularly where there is interest in schemes awarding certification or accreditation, their initiators should work with UKAS at an early stage to ensure that the schemes are sufficiently rigorous to meet EU and UK accreditation requirements. "

The new Standard (ISO 15189) has been adopted by UKAS. UKAS appoints assessing organisations on the basis of their competence to inspect pathology services for conformance to the Standard.

It is the UK government’s belief that conformity assessment and accreditation are important parts of the nation’s quality infrastructure. Ministers argue that this provides confidence to commissioners and patients and that conformance will make a significant contribution to the economy.

Some disconcerting evidence

Wilson et al conducted a meta-audit of ISO 17025 accreditation (conformity) inspections in public health laboratories. 333 cases of conformity assessment, conducted between 2000 and 2013, were studied. The study focussed on inspection findings and the effect of non-compliances on the quality of service. In simple terms the study revealed that almost all non-compliances were inconsequential; they related to matters of documentation as required by the Standard and, more importantly, almost none of these non-conformances had any likely effects on service quality.

Inspectors were most frequently found to detect what the authors describe as very minor differences between written procedures and actual practice, exemplifying a focus on documentary and procedural minutiae. Typically the non-conformances related to media production, staffing and training records, administrative errors, equipment labelling and were mostly concerned with minor textual amendments to standard operating procedures and document control. Typically inspectors would speculate about adverse events that might occur – what could go wrong, not what was going wrong.

In further support of this finding the authors report these non-conformances had diminished over the first two years of conformance assessment, indicating people were improving their understanding of what inspectors wanted to see, which often seemed ridiculous, even bizarre. Another common non-compliance was failing to encourage suppliers either to register to the Standard or, often, to forward their certificates of conformance.

The conclusion of the study is unequivocal: ‘… we should seriously question whether any genuine quality is brought into existence by this form of inspection at all’. There is no reason to assume we would have different findings with ISO 15189; early indications of inspection in practice are consistent with the findings above; pathology services that have been inspected against ISO 15189 have experienced trivial non-compliances – things that have no bearing on the quality of the work – demonstrating scientific illiteracy in inspectors as well as creating unnecessary costs.

Some general conclusions:

1. Conformance to ISO 15189 has no bearing on quality

As I shall show below, the opportunities for significant quality improvement lie beyond the Standard’s requirements. ISO 15189 is not a ‘clinical audit’, something with which pathologists are familiar, it is an audit of management practice based in a discredited theory of management (see later).

2. Conformance drives up costs

There is a series of issues driving up costs:

Buying copies of the Standard: You might have thought that Standards would be published free on the internet, but they are sold at astonishingly high prices to an audience obliged to buy. It has been reported that the cost of copies of ISO 15189 purchased by NHS Scotland was around £2.5M. The cost of purchasing the Standard and it’s associated ‘normative documents’ was £1,000 for North Devon’s pathology services. For larger laboratories the costs will be higher. Whichever way you estimate them, the costs of purchasing the necessary documents will be high.

Many laboratory services employ extra personnel for the sole purposes of creating and managing documentation and carrying out internal inspection and other duties in accordance with the Standard’s requirements.

The costs associated with bureaucratic impositions on all work activity (record-keeping for infinite purposes).

Demoralisation associated with the sheer waste of energy required to comply with requirements of questionable value and from being judged in apparently arbitrary fashions by inspectors. Compliance has a deleterious psychological impact. The inspector’s role is to find fault; any adverse reports will damage confidence in all parties and generate inappropriate actions driven by the requirements to ‘conform’.

The greatest cost is the cost associated with hiding the means for improvement.

In direct contradiction of UK government policy, conformance with ISO 151898 won’t give confidence and will impede rather than facilitate improved economic performance.

UK Government policy recognises that if applied incorrectly conformity assessment can be a burden, create barriers to trade, inhibit innovation and confuse the market. At least three of these adverse consequences occur when conformity assessment is applied ‘correctly’.

UK Government policy also states that conformity assessment schemes must facilitate, not discourage, innovation. As I shall show below, the levers for significant innovation are beyond the Standard’s requirements.

‘Quality’, within this and other standards, is conceived as a process of ‘specifications and inspection’. It is to fail to understand quality theory. It was as long ago as the 1950s that the quality gurus taught us you can’t inspect quality in.

A misguided belief

The first politician to be taken in by the idea of specifications and inspection representing ‘quality’ was Margaret Thatcher, who was persuaded that BS 5750 contained the ‘Japanese Miracle’; nothing could be further from the truth. The waves of discontent expressed by people in private-sector organisations obliged to seek accreditation to BS 5750 led Michael Heseltine to try harder with one sector – software development – which gave rise to the ‘Tickit’ scheme. In case one wonders what impact that had, consider the fact that 90% of all major IT projects fail.

Missing the opportunity to review the value-in-use of BS 5750 the UK Government worked with the International Standards Organisation to broaden the promulgation of the Standard as a requirement to trade. BS 5750 became ISO 9000, the first management standard, not a measurement or technical standard.

A better understanding of Quality

Deming, regarded as the father of quality thinking, taught us that Quality is concerned with knowledge and that knowledge can only be gained by understanding and managing the organisation as a system. The knowledge gained leads to profound performance improvement, where actions can be predicted to lead to improvement. This philosophy underpinned the ‘Japanese Miracle’ and was exemplified by Toyota’s success in its Japanese manufacturing organisations, more than 60 years ago, where factories were developed to produce vehicles to consumer demand. Control of operations was in the hands of those doing the work; the worker was his own inspector – there was no independent inspection; manufacturing lines accommodated a variety of products; movement of materials worked at the heartbeat of consumer demand. The Toyota System was the first ‘pull’ manufacturing system – take an order, make the car. The impact on quality was legendary and costs fell dramatically as a consequence.

To return to Deming, he showed that Quality methods generate a virtuous cycle; as quality improves, costs fall, as costs fall prices can be lowered; hence market-share increases and, as a consequence, companies grow; creating more jobs.

Quality in practice: North Devon as pioneers

The virtuous cycle Deming described has been evidenced in Pathology services in North Devon. My organisation’s contribution has been to help leaders with the principles to follow in order to study and improve the services. Studying pathology services as systems reveals substantial opportunities for improvement.

As with any transactional service the greatest lever for quality improvement is an understanding of demand. Pathology services – if we assume that the findings in North Devon are representative – exhibit a high level of what I describe as ‘failure demand’ (demand cause by a failure to do something or do something right for the customer).

In pathology services failure demand is better described as ‘unnecessary demand’. Unnecessary in the sense that either the test is not required in order to make informed decisions about patient care or requests for tests are inflated by the failure to take the right action on the first transaction. A further problematic type of demand can be labelled ‘unclean demand’; these are requests for tests that contain insufficient information about the context of the patient or clinical decision.

Some examples:

40% of liver function tests in patients with cardiovascular disease were found to be unnecessary. NICE guidance is for a test at the beginning of statin treatment and from there no testing is required unless dictated by a clinical need. Studying revealed a lack of attention to detail with these patients, most patients being treated by staff with little clinical expertise. This may account for the growth in the volume of testing recently experienced. Studying also revealed over-testing to have caused significant harm to a large numbers of patients.

Studying testing of urine culture revealed a doubling in specimen numbers over the last decade. Much of this occurred for similar reasons to the expansion of liver function testing, where healthcare personnel were testing as a matter of routine, for reasons they did not comprehend, with inappropriate application of other guidelines that were also only partially understood. This over testing of a low risk population disproportionately increases the number of false positive results, which devalues the quality of the test. Misinterpretation of results then leads to the inappropriate prescription of antibiotics, feeding an even bigger cycle of failure demand (antibiotic resistance) on a public-health scale.

Studying the testing of wound swabs revealed that simple venous ulcers were often taking excessive time to heal. This was leading to considerable morbidity, including social isolation. Ultimately it is also possible that slow healing is linked to a number of more serious sequelaue, including limb amputations. In addition, the burden of slow healing wounds on nursing care is large: studying revealed that in a typical GP practice of around 10,000 patients, wounds were taking (on average) about 56 weeks to heal. This required an additional session of nursing time each week just to dress the wounds (a 10% increase in nursing time for the practice).

The primary cause was a focus on productivity in the upstream healthcare service, driving workers to see volumes of patients rather than taking the time required to assess the nature of the wound and take the correct action. Swabs were a simple way to deflect the need to do the right thing, which is a relatively time-intensive measurement of blood pressure that could not be easily fitted in to the standard appointment system.

Each of these studies led on to an improved design. Improving the assessment of wounds resulted in much faster healing times and a dramatic reduction in subsequent appointments. In North Devon, 80% of wounds now heal in 12 weeks, which has removed 21 nursing appointments each week. Improving guidance on the use of urine culture, targeted at healthcare assistants who do this work, has led to the volume of urine culture requests being cut in half, reducing the potential for result-driven iatrogenic harm. Liver function tests have dropped by up to 40% by the simple introduction of evidence-based electronic ordering for chronic disease monitoring.

As a consequence of these and other quality improvements, costs have fallen dramatically. And this is a work in progress, as further services are studied, further improvements will follow.

North Devon’s work illustrates the opportunity quality thinking provides. A systems perspective takes a high-level view of (a) how tests are used in diagnostic work, (b) the laboratory processes and (c) the actions that result from testing – assessing achievement of purpose. ISO 15189 focusses with a ‘document procedures’ emphasis on (b) whereas the leverage for improvement comes from a focus on (a) and (c|); improving the diagnostic questions to address and diminishing the downstream costs of further investigations and unnecessary referrals. In systems terms this to think ‘outside-in’, to focus on ‘clean in’, ‘clean through’ and ‘clean out’. The consequences are an improvement in service quality, an increase in capacity and a reduction in operating costs (the true relationship between cost and quality). ISO 15189 will only hinder quality improvement being, at best, a burden on costs.

If the results of merely the first phase of improvement in North Devon were replicated across all pathology services the impact on quality and cost would be considerable.

Managing costs creates costs

It is a fundamental of quality thinking that an organisation must be managed by reference to its purpose. Clarity of purpose focusses attention on managing value and, as a consequence, costs fall. This is in direct contrast to Government’s approach to public services, which focusses on managing cost, in particular the costs of transactions.

The risk of focusing on transaction costs is most deleterious when services are out-sourced. The vast majority of out-sourced contracts are established on the basis of transaction volumes and their associated costs, thus there is no incentive for the supplier to reduce demand (as failure demand rises, so does the supplier’s revenue). ISO 15189 inspectors who treat the failure to oblige suppliers to register to the Standard exacerbate this problem.

But it’s the law

EU legislation and UK government policy are driving the adoption of ISO 15189. UKAS is charged with creating an ‘open market’ for inspection services, but those who provide the service are chosen on the basis of competence to inspect against the Standard.

The NHSE business plan for 2014-15 to 2016-17 includes a ‘key deliverable’ to “ensure more than 70% of all scientific and diagnostic services are part of accreditation programmes and demonstrate robust quality assurance measures by end of March 2015”.

The law governing the promulgation of ISO 15189 provides for UKAS to raise a formal objection to the Standard, and UKAS can, for a sum of money, assess any alternative quality scheme.

Getting quality improvement in pathology, realising the ministerial ambition to make a significant contribution to the economy, will be a bit of a challenge.

---

John Seddon first wrote about the damaging effects of ISO in 2000 in ‘The Case Against ISO 9000: How To Create Real Quality in Your Organisation.

John Seddon is the leader of the Vanguard organisations, now working in nine countries, helping service organisations change from a conventional ‘command-and-control’ design to a systems design. John has received numerous academic awards for his contribution to management science and won the first Harvard Business Review / McKinsey Management Innovation Prize for ‘Reinventing Leadership’ in 2010. He is the author of ‘The Whitehall Effect’.