Mechanisms of measuring key performance indicators in the pre-analytical phase

Introduction

Since pre-analytical errors first started to appear in the literature in relation to laboratory medicine in the 1970s (1-3) interest in the area has grown. Initially this growth was slow but over the last 10 to 20 years interest in this area has increased exponentially (3). The pre-analytical phase covers a large part of the total testing process (TTP) from the actual conception of the test through to the point at which the sample is placed on the analyser (4). It has been well documented that errors in the TTP are in the range of 0.05% to 10% of which around 60% are attributable to the pre-analytical phase (5,6). Within the pre-analytical phase, it is often further broken down into the pre-pre-analytical phase and the pre-analytical phase. The pre-pre-analytical phase covers the process from test conception and requesting through to phlebotomy, followed by the pre-analytical phase which includes the transport and sample processing stage (7).

Quality in healthcare has always been a well scrutinised parameter with Laboratory Medicine being at the forefront of it, especially because of the ease with which it can generate evidence. In the UK, the Francis report emphasised the need to ensure errors in healthcare are kept to a minimum (8). This was followed by the Barnes report which emphasised the need to log and monitor errors in a transparent manner as essential for good laboratory service (9). Laboratory compliance with this drive is monitored via external organisations that accredit laboratories such as UKAS (United Kingdom Accreditation Service) in the UK. These accreditation bodies ensure laboratories adhere to ISO 15189:2012 (10). The move to this standard has meant the laboratory is obliged to cover the extra-analytical aspects. That is to say the laboratory has a responsibility to monitor and improve all aspects of the TTP from request to interpretation. This means laboratories must have a mechanism to measure performance in these areas because, as Sir William Thompson said, “If you cannot measure it, you cannot improve it”.

There are certain practices and procedures that laboratories should have in place to ensure that they are well equipped to both provide the best possible quality and to respond when a problem in the TTP is identified. Central to this should be a Quality Management System which includes a quality management policy. This should include what documentation there is and how the documentation is presented and controlled. It will detail mechanisms of ensuring both staff competency and assay quality (via internal and external quality control and audit). There will also be a list of key performance indicators (KPIs) including what they are and how they are used in addition to how errors, mistakes, deviations from procedure, accidents and any other incidents whether or not they affect the patient or staff are recorded. There should be a pathology wide lead for quality with departmental quality leads and teams sitting below this. From the perspective of sample quality, it would be ideal to have control over sample transport and phlebotomy. This will aid any responses to identified quality gaps in the TTP.

It is important that the laboratory is able to engage with its users and staff (including phlebotomy staff) to provide education into why sample quality is important, how poor quality occurs and what the impact is in terms of both cost and the patient pathway. The quality team should have a mechanism to feedback any quality issues to an appropriate level with its users to ensure change is understood and implemented. In some situations, and healthcare systems penalties for repeated failure to address quality issues may be considered appropriate.

What should we be measuring?

Having established that we have a responsibility to measure pre-analytical KPIs the question arises as to what exactly should we be measuring? The International Federation for Clinical Chemistry Working Group on Laboratory Errors and Patient Safety (IFCC-WG-LEPS) has for many years strived to establish and promote what pre-analytical KPIs we should be measuring. These KPIs have evolved over many years, initially starting out at around 25 before expanding to a more exhaustive list (11,12). From this long list a conference in 2013 then reduced these down to a more manageable and practical number and ranked them in terms of priority from those that should be done and should be relatively easy to collect through to those, that whilst they should be done may be more difficult in practice to assess (13). The recommended KPIs cover the whole of the pre-analytical phase from the appropriateness of the request through to the quality of the sample once it has arrived at the laboratory and are listed in Table 1.

Considerations in KPI selection

Despite this information provided by IFCC-WG-LEPS the ability to perform the recommended KPIs varies greatly depending on local set up including staffing levels, IT capabilities and laboratory equipment. A recent UK survey by the ACB Pre-Analytical Phase Working Group study showed that huge variations in Laboratory Information Management System (LIMS) providers coupled with differences in equipment means there is inevitably a huge variation in what can be measured and how it is done (14). The question how will be discussed below, but in terms of what is measured there are certain points that need to be considered.

The first stage is to pick the most appropriate KPI. Table 1 outlines some recommendations but there may be other more important local KPIs. There must be a robust way of measuring the KPI and possible methodologies will be discussed below. But just as important as the how it will be done, are the questions:

• How frequently it needs to be done to be useful?
• How will the data be analysed?
• What are the cut offs for action?
• What is the action?

Although ISO 15189:2012 stipulates that pre-analytical KPIs must be measured it does not state what needs to be done with them. The key reason for measuring KPIs is to improve the patient experience. To facilitate this, an SOP (standard operating procedure) should be written up that also includes a mention of the format of the data and suggested actions. Measuring pre-analytical KPIs would be of little use if they are not fed back to the appropriate users to drive improvement. Actions should involve the relevant users in problem locations and to aid this, data must be presented in a clear and unambiguous manner that can be easily understood by all.

Mechanisms of measuring quality indicators?

Once a KPI has been chosen there needs to be a robust mechanism of collecting data. There are various ways to do this which are discussed below in increasing order of recommendation

Non-conformances (NC)

Logging of NC or incidents with the laboratory is an important part of the quality management systems in place within the laboratory. NCs are recorded when errors are detected in the laboratory. The use of NCs as a way of monitoring KPIs is a possibility but has some key disadvantages. It does not involve a process of actively looking for KPIs but logs those that are identified as part of routine working or following conversations with users. Examples are many but include, failure to process a sample correctly, insufficient identifiers, or if the sample was collected from the wrong patient. It is retrospective and data collection is infrequent and prone to under reporting of errors if used in isolation. This has an impact on its effectiveness regarding timely feedback to users. In busy laboratories this frequency is not likely to be robust. Inherent human error factors mean that the NC mechanism could miss many incidents. It is also not ideally suited to pre-analytical KPIs as they are generally too frequent in occurrence, so the NC route is only likely to be useful if there is a sudden unexpected but significant problem with several samples from a certain location. In this situation NCs fill a very important gap in that they are a place to log laboratory incidents or errors that are not significant enough to require a staff or patient safety incident to be logged via DATIX (DATIX, London) Longer intervals for NC trend analyses may mean identification of issues long after the event and is less than satisfactory from a user perspective. For these reasons NCs are better used as a mode of logging significant changes from a certain location in a pre-analytical KPI to be used to then document follow up of the issue.

Audit

Audits are also an essential part of maintaining quality in the laboratory and all laboratories should have an audit program to cover all aspects of the TTP. The audit process should be a well embedded part of the laboratories quality management system. The audit approach can be applied to pre-analytical KPIs and is for some parts of the process, the only realistic option. Audits in the laboratory usually take the form of horizontal or vertical audits. For pre-analytical KPIs this would usually be a horizontal audit looking at a cross section of data relating to a particular step in the TTP. For example, investigating data entry errors can only be done via a horizontal audit by looking at for example 100 forms in the booking in process for data entry errors. The disadvantages of audits are that it is a manual process and can be time consuming. Given the current pressures on laboratories, audits are often delayed and therefore there is a risk that the robustness of the process will not be as good as more automated methods. Audits also rely on manual processes by human auditors and are therefore susceptible to human error. Hence the data will not be as reliable as that collected by automated mechanisms. Data collection is retrospective so cannot address the issue in real time. The frequency of audits of a specific area may affect the speed at which the non-conformances are picked up and sometimes may be missed entirely if the events are intermittent.

Manual logging

Manual logging is the process of manually entering KPI data into a database (often written first then on a computer later) as the errors occur, e.g., noting missing information from request forms or logging lipaemic samples. The manual logging of KPI errors into a live database is the best solution for developing laboratories that do not have robust IT systems from which data can be extracted. This is a real time process of recording data but may be retrospective depending on when the data is entered into the electronic database. Due to the fact it involves people it will inevitably under report issues and may be user dependant. It is also less conducive to the collection of multiple errors. The data may need to be transferred to a computer for analysis if it is not directly entered and this may delay the process. So, whilst this is a good solution, it is not quite real time in terms of feedback and data collection may not be all that robust.

Equipment software

Using the middleware software that comes with the laboratories’ main equipment is one good way of collecting the data. Error codes generated by this software should transfer to the LIMS and then back to the users, so they get real time feedback of issues with their requests for example when samples are haemolysed or if incorrect samples are sent. In order for this method to be useful there are several methods of recording an error that will be discussed under LIMS below. The data also needs to be extractable so that retrospective audit of the errors is possible.

LIMS

As with the middleware solution above, the use of LIMS is one of the best ways to collect pre-analytical KPIs. It automates the data collection and for some markers such as haemolysis it can take all human judgement and error out of the process. Manual entry of errors may still be needed in some instances. As described above and later, there needs to be a robust mechanism of retrieving data and processing it to produce meaningful results that can be fed back to users and the laboratory. There are various ways of using the LIMS and middleware as a data entry point for KPI data.

• Firstly, one can use a completely separate test code for errors and then enter an error code relating to the error type against that. This is useful for errors that affect whole samples;
  • e.g., error code → no sample received
• Secondly you can just enter a coded comment result against the affected test. This has the benefit of informing the requestor and producing something that is useable for data extraction;
  • e.g., haemolysed sample
• Finally, you can have the error as a comment against the test. This may be useful where there is an error but also a result affecting specific tests only. These comments should be coded and therefore standardised and should be extractable.

By using this automated mechanism of error logging, extraction of data can then be easily done by searching for specific test codes, test results or by key word extractions. Extracted data can then be processed using a database or spreadsheet as discussed below.

Conclusions

The article presented here has shown that there are clear drivers from international standards to promote the collection of KPIs in the pre-analytical phase. The laboratory is uniquely placed to record and access this data and therefore has a responsibility to do so. Whilst ISO 15189:2012 only stipulates the need to collect data, this is clearly a pointless exercise if there is no feedback into the system to improve the TTP and therefore, improve the patient pathway. It is important to have SOPs clearly defining what the process is, the frequency of evaluation, how the data is fed back to users. Most importantly, the action limits and details of actions that need to be taken if these limits are exceeded should be clearly elucidated.

In the opinion of the authors although the most important aspect of frequency is that it is defined and robust It must all so be of sufficient frequency that the KPI in question can be addressed in an appropriate time frame to avoid allowing poor quality to perpetuate. It is our opinion that KPIs should be collected on a monthly basis. Once collected, how the data is analysed is also important. The analysis must follow a robust mechanism that minimises manual processes as much as possible. Ideally this whole process could be managed by laboratory software, either the LIMS or middleware associated with the equipment which should be capable of presenting the data in various visual formats. The other alternative is to set up excel documents to manipulate the data once extracted and convert it into an easily presentable format. Currently this method is the most commonly available option, but the laboratories have a responsibility to put pressure on laboratory software suppliers to build the required tools into their future product releases. The defining of cut offs and action mechanisms for each KPI is beyond the scope of this manuscript but each KPI should have a cut off and action mechanism that is defined and agreed with the users. Feedback must include the users and must not just be retained internally because as said above there is no point collecting the data if it’s not going to be used to affect change.

The gold standard to extract this data is to automate it as far as possible using LIMS. When tendering for new LIMS, the laboratory should be asking for the provider to build in mechanisms to facilitate collection and processing of data. The ideal would be to have an automatically produced real time dashboard of pre-analytical KPIs to allow instant identification of issues in the pre-analytical phase.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Journal of Laboratory and Precision Medicine for the series “Reducing errors in the pre-analytical phase”. The article has undergone external peer review.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jlpm.2018.05.06). The series “Reducing errors in the pre-analytical phase” was commissioned by the editorial office without any funding or sponsorship. Michael P. Cornes served as an unpaid Guest Editor of the series and serves as an unpaid editorial board member of Journal of Laboratory and Precision Medicine from December 2016 to November 2018. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Lippi G, Simundic AM, Mattiuzzi C. Overview on patient safety in healthcare and laboratory diagnostics. Biochem Med 2010;20:131-43. [Crossref]
2. Pleban M, Carraro P. Mistakes in a stat laboratory: type and frequency. Clin Chem 1997;43:1348-51. [PubMed]
3. Cornes MP, Church S, van Dongen-Lases E, et al. The role of European Federation of Clinical Chemistry and Laboratory Medicine Working Group for Preanalytical Phase in standardization and harmonization of the preanalytical phase in Europe. Ann Clin Biochem 2016;53:539-47. [Crossref] [PubMed]
4. Plebani M, Laposata M, Lundberg G. The Brain-to-Brain Loop Concept for Laboratory Testing 4Years After Its Introduction. Am J Clin Pathol 2011;136:829-33. [Crossref] [PubMed]
5. Plebani M. Errors in clinical laboratories or errors in laboratory medicine. Clin Chem Lab Med 2006;44:750-9. [Crossref] [PubMed]
6. Kalra J. Medical errors: impact on clinical laboratories and other critical areas. Clin Biochem 2004;37:1052-62. [Crossref] [PubMed]
7. Plebani M. Exploring the iceberg of errors in laboratory medicine. Clin Chim Acta 2009;404:16-23. [Crossref] [PubMed]
8. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry Chaired by Robert Francis QC. February 2013.
9. Barnes I. Pathology quality assurance review. Jan 2014. Available online: https://www.england.nhs.uk/wp-content/uploads/2014/01/path-qa-review.pdf
10. ISO 15189:2012. Medical laboratories -- Requirements for quality and competence. Available online: https://www.iso.org/standard/56115.html
11. Sciacovelli L, Plebani M. The IFCC Working Group on laboratory errors and patient safety. Clin Chim Acta 2009;404:79-85. [Crossref] [PubMed]
12. Plebani M, Sciacovelli L, Marinova M, et al. Quality indicators in Laboratory Medicine: A fundamental tool for quality and patient safety. Clin Biochem 46 2013;46:1170-4.
13. Plebani M, Sciacovelli L, Aita A, et al. Harmonisation of preanalytical quality indicators. Biochem Med (Zagreb) 2014;24:105-13. [Crossref] [PubMed]
14. Cornes MP, Atherton J, Pourmahram G, et al. Monitoring and reporting of preanalytical errors in laboratory medicine: the UK situation. Ann Clin Biochem 2016;53:279-84. [Crossref] [PubMed]