A current analysis of quality indicators in Chinese clinical laboratories

Introduction

Information provided by Clinical laboratories are essential in the delivery of healthcare in almost all healthcare settings, including preventative health, diagnosis and monitoring of health status and treatment and in prognosis. Therefore, the activities of the laboratory including diagnostic errors, timeliness of delivery of results and efficiency of the laboratory processes directly impact healthcare delivery and outcomes (1,2). “Improving Diagnosis in Health Care” (3) is a report published by The Institute of Medicine which provides detailed analysis of the causes of diagnostic errors and near misses. It also provides a systematic strategy to reduce these in a timely fashion. Benchmarking against similar laboratories is a well-recognised approach to identify the key errors and monitor the outcome of any intervention (4).

Adoption of a system-based approach to improve quality of laboratory testing reduces variation and activities linked to Accreditation have been shown to improve patient safety and outcomes (5-12). However, the capability or resources to achieve accreditation against international standards such as ISO 15189 is a limiting factor for many laboratories in the Asia-Pacific (APAC) region. Thus, other surrogate improvement activities measurable and comparable against peers are necessary. There have been published reports on long term surveys providing data on quality indicators in broad areas such as quality, cost, and turnaround time (13,14). Introduction of some External Quality Assurance (EQA) schemes and Benchmarking surveys have provided laboratories with performance data which can be utilised in process improvement (15).

Reliable quality indicators (QI) in the total testing process (TTP) including Indicators of the extra-analytical phases is vital for identifying areas where improvement is needed, and these have been developed in some countries (16,17). The International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) via ‘‘Laboratory Errors and Patient Safety’’ working group of (WG LEPS) has established a relatively complete model of quality indicators (MQI) and related quality specification available on a specifically developed website (www.ifcc-mqi.com) which can be used as a benchmark by different laboratories around the world. The evaluation of these quality indicators has been performed at a global level and preliminary results have been published (18,19). Occasional surveys have also been demonstrated to achieve improvement (20). There is an Australian QI survey which has been active since 2012 and which has approximately 60% of Australian laboratories represented (21). KIMMS includes a Failure Mode and Effects Analysis FEMA) based risk score as well as frequency of error.

Improvements in healthcare is a major focus in the APAC region despite the great diversity in culture and financial capabilities. Since Laboratory Medicine plays a crucial part in medical diagnosis and treatment, improvement in this field is important for better healthcare delivery to this vast population. Roche Diagnostics sought to determine the ‘State of the Art’ and progress by surveying APAC laboratories which started in 2011 (13). These surveys gathered data on three key areas of laboratory activities mainly focussing on Clinical Chemistry and Immunology. Although the gathered data are not extensive as the Quality Indicators suggested by the IFCC, they provide useful information on performance compared with peers. National Health and Family Planning Commission of the People’s Republic of China suggested a list of 15 QIs in 2015, covering the most error-prone testing processes of laboratory medicine based on the IFCC MQI (22). The debate about the number and type of effective QIs continues (23). The results of an EQA Scheme for 15 QIs established in 2015 by the National Centre for Clinical Laboratories of China (CNCCL) have been published (24-28). However, the Roche survey covers many aspects of not covered by the CNCCL’s EQA program is more extensive. We aimed to report on the performance of Chinese laboratories compared with APAC in the 2019 Roche Survey. Comparisons were made on quality indicators on post analytical phase and laboratory activities integral to improvement in quality and safety. These included participation in EQA programs, Accreditation against an international standard, Continuous Quality Improvement activities, Key Performance Indicator (KPI) measurement, TAT definitions and goals, and levels of automation. We present the following article in accordance with the SURGE reporting checklist (available at https://dx.doi.org/10.21037/jlpm-21-19).

Methods

The benchmarking survey started in 2011 by collecting feedback from clinical laboratory managers and directors on their laboratories’ operation and performance. The questionnaire is provided in the Supplementary data. There were 181 laboratories from 12 countries/regions participated back then. The survey is carried every alternate year, and benchmarking report will be released to the participants for their reference.

In August 2019, a new edition of the survey was launched via an online platform (www.labinsights.com). Participants from APAC can freely access the platform using any internet browser and complete the survey digitally. Participants can also access the benchmarking survey report afterwards on the same platform at any time once it is ready. The questionnaires were distributed by Roche affiliates in hard copy as well for certain countries which translated version is required.

Statistical analysis

The results presented are the data obtained from the survey in 2019. In total, there were 1,158 laboratories from 17 countries/regions in APAC answered the survey. Twenty one percent of the laboratories were from developed countries (using the IMF World Economy Outlook 2018 (https://www.imf.org/en/Publications/WEO/Issues/2018/09/24/world-economic-outlook-october-2018). Fifty nine percent were government laboratories, 32% were private hospitals and 9% were private commercial organisations. Approximately 29% were small (<250 samples per day), 45% were medium sized (251–1,000 samples per day) and 27% were classified as large laboratories (>1,001 samples per day). Surveys were received from the following countries (Table 1). Details of the participant laboratory types and sizes are shown on Table S1. No incentives were offered.

Table 1 Participating countries/regions
Full table

Results

We will report on the results by key areas of quality, speed and productivity for Chinese laboratories compared with the overall APAC group data. Further details are available in the Supplementary data provided.

Quality—Accreditation

In total 33.1% of the surveyed laboratories in China were accredited by an external agency compared with 40.3% in APAC with the majority having ISO 15189 accreditation in both China and overall APAC groups. Most Chinese laboratories who participated in the survey were accredited by national or provincial agencies rather than international agencies. More specifically 66.8% and 77.3% of survey participants had accreditation with National Centre for Clinical Laboratories (NCCL) and Province/City Centre for Clinical Laboratories (PCCL) respectively. Interestingly only 34.8% of Chinese laboratories with international accreditation used international EQA and 74.2% of these laboratories had a continuous improvement program for accreditation. Overall, only 15.8% of Chinese laboratories in the survey used international EQA programs compared with 43.2% in APAC, with EQAS and College of American Pathologists (CAP) being most popular in China. Although many Chinese laboratories (79.7%) did not use or plan to use international EQA, the majority did participate in national EQA programs. Figure 1 shows the participation in external accreditation and EQA programs.

Figure 1 Participation in external accreditation and EQA programs.

Quality—continuous improvement

As with the APAC region, focus on continuous improvement is evident in China. 57.4% of Chinese laboratories in the survey had a continuous improvement team compared with 68% in APAC. In the 2015 Survey, 61% of APAC laboratories had a continuous improvement team, so the trend is increasing. Overall, the utility of several continuous improvement activities is slightly lower than APAC. Detailed data on types of Continuous Improvement Activities and their frequencies are shown in Figure 2.

Figure 2 Details of continuous improvement activities.

Quality—key performance indicators

A variety of KPIs were used by the surveyed laboratories which included measures on quality, productivity and satisfaction. The use of KPI and frequency of its use were: Turnaround Time (TAT) (78.9%), Performance in EQA program (75.2%), Customer satisfaction (48.6%), Cost reduction (71.2%), Employee satisfaction (46.4%), Employee productivity (40.6%), Work-space utilisation (27.3%), sigma metric calculation (25.3%) and sample rejection rate (68.7%. Overall, the measured KPIs were comparable with the APAC group (Figure 3).

Figure 3 Comparison of KPIs measured by participation APAC and Chinese laboratories.

Sample quality

A variety of methods were used to check for sample quality including manual check (86.0%), pre-analytical instrument (44.5%) and analytical instrument (32.9%) (Figure 4). The mechanisms employed for notification of sample rejection included phone call (96.2%), QC meeting (55.2%), IT Alert system (46.1%), Instant messaging App (44.7%), SMS (29%) and e-mail (8.3%). Interestingly, the use of instant messaging app for sample rejection notification was higher (44.7%) compared with the APAC group (29.1%) (29). Figure 4A and B show the various methods utilised for sample quality check and sample rejection notification.

Figure 4 Methods utilized for sample quality check and sample rejection notification. (A) Figure gives data where the sample quality check is conducted. (B) Figure provides details on how sample rejection is transmitted to the referrer.

Quality—IT

Laboratory information Systems have been implemented and in use in 95% of Chinese laboratories compared with 93.8% in APAC. 38.6% of Chinese laboratories use middleware compared with 37.7% in APAC. The survey results with regards to the use of IT functions are summarised in Figure 5.

Figure 5 Comparison of use of IT functionality.

Speed—STAT sample

China invests significantly more resources for STAT samples than the APAC in term of provision of dedicated STAT laboratories, dedicated instrument, and staff to handle these specimens (Figure 6).

Figure 6 Comparison of STAT sample handling. TAT, turnaround time.

Speed—critical results

There were three-time intervals given for reporting a critical result. Although only a minority of Chinese labs in the survey 66/399 answered ‘No target’ for the questions relating to critical results reporting, 22% of the lab had a critical result notification target of <5 minutes compared with 30% in APAC (Figure 7).

Figure 7 Critical result reporting.

Speed—TAT (turnaround time)

There was variability TAT monitoring in terms of the phases of the total testing cycle monitored and monitoring for all specimens by departments. Majority of laboratories monitored laboratory TAT (89.7%) whilst total turnaround time, pre-analytical TAT, analytical TAT, and post analytical TAT were monitored by 53.6%, 48.9%, 48.9%, and 46.1% respectively. Most Chinese laboratories in the survey utilised IT function for monitoring TAT (68.5%) with 45.8% of all laboratories monitoring TAT in real time (Figure 8).

Figure 8 TAT monitoring in APAC. TAT, turnaround time.

For STAT specimens, 60% and 48.3% of laboratories have a <60 min target for clinical chemistry and immunoassay specimens respectively. For routine specimens, 63.9% and 42.9% of laboratories have a target <180 min in chemistry and immunoassay respectively. Many laboratories in China also have specific assay turnaround times; 87.5% of laboratories for cardiac markers, 79.7% for liver function test, 81% for renal function test and 73.9% for arterial blood gas. The median TAT target for cardiac markers was 60 minutes (Interquartile range: 30 to 90 minutes). Figure 9 show routine and STAT chemistry and immunoassay TATs.

Figure 9 TAT for STAT and routine samples. (A) Figure is TAT for routine chemistry. (B) Figure is TAT for STAT chemistry. (C) Figure is TAT for routine immunoassay. (D) Figure is TAT for STAT immunoassay. TAT, turnaround time.

Productivity: laboratory automation and consolidation of instruments

Approximately a third of Chinese (32.3%), and 39.8% of APAC laboratories had the chemistry and immunoassay modules consolidated onto the same instrument with 43.4% of Chinese and 33.8% of APAC laboratories having laboratory automation for pre- and post-analytical processes. Figure 10 show the comparative levels of consolidation and automation in Chinese and APAC laboratories.

Figure 10 Laboratory productivity: consolidation and automation in China and APAC. (A) Figure is instrument consolidation. (B) Figure is laboratory automation.

In terms of FTE productivity, the median number of samples per full time equivalent (FTE) was 100 compared with 92 for APAC and median number of tests per FTE was 750 compared with 533 for APAC. The median number of samples per FTE increased when instruments were consolidated (133 vs. 100) or when there was laboratory automation (143 vs. 88). The median number of samples and tests per m² in Chinese laboratories were 6 and 38 respectively. In Chinese laboratories the median number of samples and test per instrument were 107 and 750 respectively.

Discussion

The expected varied degree of compliance with the implementation of best practice has been confirmed by this extensive survey of Chinese Laboratories. However, there is a common focus on quality and continuous improvement in various aspects of laboratory practice. These are evident by monitoring of TAT, external accreditation, participation, and performance of the laboratory in EQA. China seems to have invested more resources on STAT samples compared with routine samples. This is evident in the provision of dedicated stat laboratories, dedicated instrument, and staff to handle these specimens as well as the significantly shorter target TAT set for STAT samples compared with routine samples. This suggests a segregation of work and different workflows based on urgency which is not the case in APAC overall.

Most Chinese laboratories are currently enrolled or plan to enrol in international accreditation. However, most have not or do not plan to enrol in international EQA schemes. The fact that most Chinese laboratories are enrolled in national rather than international EQA programs perhaps reflect the fact that although participation in EQA is not mandatory in China, many laboratories recognise the value in participation in a national EQA program which is more practical. However, accreditations such as ISO 15189 which represent the quality level of the laboratory is encouraged by the government which can improve hospital influence.

Most Chinese laboratories monitored laboratory TAT (89.7%), many did not monitor total TAT or individual components such as pre-analytical TAT, analytical TAT, and post analytical TAT. The reason for this may be related to IT capability. The IT software installation required for TAT monitoring are often bundled sales with instruments, especially with automation. Thus, laboratories that do not have automation may not be able to monitor TAT.

Whilst there is no significant difference between China and APAC in the proportion of laboratories having TAT target <60 min for STAT chemistry samples, this is not the case for routine chemistry samples. This is despite proportionately similar number of laboratories having adopted instrument consolidation and laboratory automation between Chinese laboratories and APAC. However, the difference may be explained by the significantly a greater number of dedicated STAT laboratories, dedicated instrument, and staff to handle STAT samples in China.

Although the percentage of laboratories with instrument consolidation was similar between China and APAC, a slightly higher proportion of Chinese laboratories seem to have laboratory automation. This may explain the higher median number of tests per FTE in China which was even higher with automation.

Our survey results provide valuable data on a variety of QIs of Chinese laboratories and covers many areas of the laboratory processes which have not been reported in the CNCCL’s EQA data on the 15 QIs (22-26).

Limitations

The major limitation to these findings is that this survey is voluntary and self-reported. Therefore, there is a possibility that the responders may be biased towards the more sophisticated laboratories. The number of participants in this survey represents a small proportion of all Chinese clinical laboratories, mainly from users of Roche instruments which may not be representative of all laboratories. However, for the individual participants it provides useful comparative data with their peers. Furthermore, here we report on a snapshot in time rather than comparison of performance over time. Thus, this survey does not allow an assessment of whether there have been improvements in QIs over time.

Conclusions

Clinical diagnostic laboratories are faced with similar challenges of increasing workloads, need for quality improvement and to achieve faster turnaround times. Comparison between Chinese laboratories with APAC highlights the fact that whilst the performance of China is generally comparable to APAC there are some differences in practice that are specific to China. This report provides a useful snapshot of the performance of Chinese laboratories in a number of benchmarking quality indicators.

This Survey reports on a variety of QIs of Chinese laboratories covering many areas of the laboratory processes which have not been reported previously. The Chinese laboratories seem to have invested more resources on STAT samples, enrol in national EQA rather than international EQA programs and have longer overall TAT targets for routine Chemistry results. This may reflect cultural differences in Chinese laboratories compared with APAC in terms of workforce utilisation and expectations of referring Chinese doctors.

Acknowledgments

We would like to acknowledge Roche Diagnostics Asia Pacific for providing the Survey data.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the SURGE reporting checklist. Available at https://dx.doi.org/10.21037/jlpm-21-19

Data Sharing Statement: Available at https://dx.doi.org/10.21037/jlpm-21-19

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://dx.doi.org/10.21037/jlpm-21-19). Dr. TB serves as an unpaid editorial board member of Journal of Laboratory and Precision Medicine from January 2019 to December 2020.The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Singh H, Graber ML. Improving Diagnosis in Health Care--The Next Imperative for Patient Safety. N Engl J Med 2015;373:2493-5. [Crossref] [PubMed]
2. Hawkins RC. Laboratory turnaround time. Clin Biochem Rev 2007;28:179-94. [PubMed]
3. National Academies of Sciences, Engineering, and Medicine. 2015. Improving Diagnosis in Health Care. Washington, DC: The National Academies Press. 10.17226/21794.10.17226/21794
4. Galloway M, Nadin L. Benchmarking and the laboratory. J Clin Pathol 2001;54:590-7. [Crossref] [PubMed]
5. Sciacovelli L, Aita A, Plebani M. Extra-analytical quality indicators and laboratory performances. Clin Biochem 2017;50:632-37. [Crossref] [PubMed]
6. Sciacovelli L, Lippi G, Sumarac Z, et al. Quality indicators in laboratory medicine: the status of the progress of IFCC working group “Laboratory Errors and Patient Safety” project. CCLM 2017;55:348-57. [Crossref] [PubMed]
7. Leatherman S, Ferris TG, Berwick D, et al. The role of quality improvement in strengthening health systems in developing countries. Int J Qual Health Care 2010;22:237-43. [Crossref] [PubMed]
8. Shin BM, Chae SL, Min WK, et al. The implementation and effects of a clinical laboratory accreditation program in Korea from 1999 to 2006. Korean J Lab Med 2009;29:163-70. [PubMed]
9. Wattanasri N, Manoroma W, Viriyayudhagorn S. Laboratory accreditation in Thailand: a systemic approach. Am J Clin Pathol 2010;134:534-40. [Crossref] [PubMed]
10. Peter TF, Rotz PD, Blair DH, et al. Impact of laboratory accreditation on patient care and the health system. Am J Clin Pathol 2010;134:550-5. [Crossref] [PubMed]
11. Tholen DW. Improvements in performance in medical diagnostics tests documented by inter-laboratory comparison programs. Accred Qual Assur 2002;7:146-52. [Crossref]
12. Chaudhary R, Das SS, Ojha S, et al. The external quality assessment scheme: five years’ experience as a participating laboratory. Asian J Transfus Sci 2010;4:28-30. [Crossref] [PubMed]
13. Badrick TC, Gutscher A, Sakamoto N, et al. Diagnostic laboratories in Asia Pacific region: investigation on quality characteristics and time of reporting. Clin Biochem 2017;50:625-31. [Crossref] [PubMed]
14. Badrick TC, Gutscher A, Chin D. Diagnostic Laboratories in India: Investigating Quality Characteristics, Productivity and Time of Reporting. Indian J Clin Biochem 2018;33:304-13. [Crossref] [PubMed]
15. Kristensen GBB, Aakre KM, Sandberg S. How to conduct external quality assessment schemes for the pre-analytical phase? Biochemia Medica 2014;24:114-22. [Crossref] [PubMed]
16. Khoury M, Burnett L, Mackay M. Error rate in Australian chemical pathology laboratories. Med J Aust 1996;165:128-30. [Crossref] [PubMed]
17. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003;15:523-30. [Crossref] [PubMed]
18. Plebani M, Sciacovelli L, Lippi G. Quality indicators for laboratory diagnostics: consensus is needed. Ann Clin Biochem 2011;48:479. [Crossref] [PubMed]
19. Sciacovelli L, O’Kane M, Skaik YA, et al. IFCC WG-LEPS. Quality indicators in laboratory medicine: from theory to practice. Preliminary data from the IFCC Working Group Project “Laboratory errors and patient safety”. Clin Chem Lab Med 2011;49:835-44. [PubMed]
20. Nakhleh RE, Souers RJ, Bashleben CP, et al. Fifteen years’ experience of a college of american pathologists program for continuous monitoring and improvement. Arch Pathol Lab Med 2014;138:1150-5. [Crossref] [PubMed]
21. Badrick T, Gay S, Mackay M, et al. The key incident monitoring and management system - history and role in quality improvement. Clin Chem Lab Med 2018;56:264-272. [Crossref] [PubMed]
22. National Health and Family Planning Commission. Notification about the publishing of quality control indicators. Available online: http://www.nhfpc.gov.cn/yzygj/s3586/201504/7cfcbeb2f01745e f94a1ce57a4497f4c.shtml. Accessed:
23. Meier FA, Badrick TC, Sikaris KA. What’s to be done about Laboratory Quality? Process Indicators, Laboratory Stewardship, the Outcomes Problem, Risk Assessment and Economic Value: Responding to Contemporary Global Challenge. Am J Clin Path 2018;149:186-96. [Crossref] [PubMed]
24. Duan M, Kang F, Zhao H, et al. Analysis and evaluation of the external quality assessment results of quality indicators in laboratory medicine all over China from 2015 to 2018. Clin Chem Lab Med 2019;57:812-21. [Crossref] [PubMed]
25. Duan M, Ma X, Fan J, et al. National surveys on 15 quality indicators for the total testing process in clinical laboratories of China from 2015 to 2017. Clin Chem Lab Med 2018;57:195-203. [Crossref] [PubMed]
26. Fei Y, Kang F, Wang W, et al. Preliminary probe of quality indicators and quality specification in total testing process in 5753 laboratories in China. Clin Chem Lab Med 2016;54:1337-45. [Crossref] [PubMed]
27. Fei Y, Zeng R, Wang W, et al. National survey on intra-laboratory turnaround time for some most common routine and stat laboratory analyses in 479 laboratories in China. Biochem Med (Zagreb) 2015;25:213-21. [Crossref] [PubMed]
28. Fei Y, Zhao H, Wang W, et al. National survey on current situation of critical value reporting in 973 laboratories in China. Biochem Med (Zagreb) 2017;27:030707 [Crossref] [PubMed]
29. Badrick T, Saleem M, Wong W. Turnaround times and modes of reporting critical results in Asian laboratories. Ann Clin Biochem 2021;58:247-50. [Crossref] [PubMed]