Evaluation of the analytical performances of six measurands for thyroid functions of Mindray CL-2000i system
Introduction
Thyroid stimulating hormone (TSH), or thyrotropin, is a hormone of about 28 kDa, produced by the thyrotrope cells in the anterior pituitary glands. The production and the delivery in blood of TSH stimulates, in turn, the thyroid gland to produce the free thyroxine (FT4) and the free triiodothyronine (FT3), two tyrosine-based hormones, that are primarily responsible for the regulation of metabolism. The precursor of FT4 and FT3 is thyroglobulin (Tg), a huge 660 kDa dimeric protein that acts as a substrate for the synthesis of thyroxine and triiodothyronine as well as the storage of the inactive forms of thyroid hormone and iodine. The latter is oxidized by thyroperoxidase (TPO) before being additioned to Tg tyrosine residues.
Thyroid gland disorders are among the most common endocrine conditions evaluated and treated by clinicians. Indeed, thyroid dysfunction, initially mostly asymptomatic, may evolve into biochemical physiological changes to clinically symptomatic diseases (1). Because of the lack of specificity of the typical clinical manifestations, the diagnosis of hypothyroidism is based primarily upon laboratory testing (1,2). Therefore, TSH determination has become a quite frequent test requested by physicians’ prescription, especially for assessing subclinical hypothyroidism (3). In autoimmune thyroid disease (AITD), the measurement of thyroid autoantibodies, such as anti-TPO (TPOAb) and anti-Tg (TgAb), are utilized for diagnosis, being the main thyroid autoantigens involved in Hashimoto’s thyroiditis (chronic lymphocytic thyroiditis) and Graves’ disease (4).
The measurement of TSH circulating levels is routinely performed in clinical laboratories by using automated platforms. The full automation confers many analytical advantages, such as good precision, rapid measurements and good sensitivity. Ideally, data on performances of an instrument should be made available before the implementation into the laboratory. Indeed, manufacturers, collect a lot of evidences regarding the instrumental performances for precision, linearity, trueness, etc. usually estimated by rigorous protocols such as that suggested by the Clinical and Laboratory Standards Institute (CLSI). Nevertheless, the implementation of a quality system complying with the ISO 15189:2012 calls for the verification or validation of analytical methods before the utilization of instruments for routine analyses (Sections 5.5.1.2 and 5.5.1.3) as the means of guaranteeing that their characteristics meet the specifications obtained during the manufacturers’ validation (5,6). Another aspect to consider is that all immunoassays remain sensitive to endogenous interferences that may cause analytical errors, threatening clinical decision making and patient safety. As example, in biotin-streptavidin immunoassays, elevated endogenous biotin due to therapeutic treatment of food supplementation can result in falsely elevated or depressed hormone levels in competitive or non-competitive sandwich assays, respectively (7).
The aims of this study were to evaluate and verify the analytical performances of the new Chemiluminescence Immunoassay (CLIA) System CL-2000i from Mindray, that include the analyses of TSH, FT4, FT3, Tg, TPOAb and TgAb for the diagnosis of thyroid disorder.
Methods
Assays description
The Mindray CL-2000i is a chemiluminescent system, featured by: (I) high throughput, up to 240 tests/hour; (II) flexible connections for lab automation; (III) a large operational capacity, up to 300 samples in one batch and supporting continuous loading; (IV) an intuitive and easy software interface. CL-2000i system utilizes micron superparamagnetic particles platform with alkaline phosphatase (ALP) labeled reagents and AMPPD substrates. CL-2000i allows performing several tests, such as tumour marker, bone marker, cardiac marker, hormones and tests for fertility, thyroid and infectious disease. TSH represent a third-generation assay with a functional sensitivity of 0.01–0.02 mU/L. TSH and Tg are two-site sandwich assay. FT3 and FT4 are biotin-streptavidin competitive CLIA, while TPOAb and TgAb are biotin-streptavidin two-steps non-competitive assays. CL-2000i presents a traceable method for TSH (WHO 81/565), for Tg and TgAb (BCR CRM 457).
Precision study
Precision was evaluated by utilizing the Mindray Thyroid function multi internal controls materials (IQC) (Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China) for TSH, FT3, FT4, Tg, TPOAb and TgAb. The Bio-Rad Multiqual 1,2,3 Unassayed (Bio-Rad, Hercules, CA, USA) was used as third part QC material for evaluating precision of THS, FT3, FT4, Tg. Precision estimation was performed by evaluating triplicate measurements of aliquots of the same samples, performed for a total of five non-consecutive days. Analysis of variance (ANOVA) was used to estimate both repeatability (intra-day precision) and within-lab (total) precision, by using the CLSI EP15-A3 protocols (8). The manufactures’ claims for precision were compared against precision results obtained for the IQC materials and verified by using the method suggested by EP15-A3.
Method comparison study and matrix effect evaluation
A total of 150 (serum and plasma) specimens, covering the most clinical relevant range of TSH, FT3 and FT4, Tg, TPOAb and TgAb were collected. The following analytical systems were compared with respect to Mindray CL-2000i system: (I) Roche Cobas 6000 (e601) (Roche Diagnostics, Basel, Switzerland) for TSH, FT3 and FT4; (II) Beckman Coulter Access Immunoassay system (Beckman Coulter Diagnostics Division Headquarters, Brea, CA, USA) for Tg, LIAISON Diasorin (Saluggia, Vercelli, Italy) for TgAb and TPOAb. Paired results were evaluated for outliers by the Grubbs test and following the Passing-Bablok regression was used to estimate constant and/or proportional bias.
Further, for a total of 40 serum specimens, a corresponding lithium-heparin plasma aliquot was available. This allowed to test whether TSH Mindray CL-2000i assay presented a matrix measurement bias.
Limits of detection and linearity
The linearity test was performed using a series of serial dilutions (1/2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256) with dilution sample by manufacture.
The detection limit was calculated by repeating a series of pools at known concentrations (for Tg from 0.4 to 0.00 µg/L and for TSH from 0.1 to 0.000 mUI/L) for 10 consecutive days and 2 times in the same series.
Statistical analyses
Statistical analyses were performed by Analyze-it (v2.07), MedCalc Statistical Software version 18.2.1 and by R software for statistical computing v3.4.3 (https://www.R-project.org/). Variance decomposition by ANOVA was performed by ANOVA using an in-house script developed for estimating all the steps suggested by EP15-A3 and for verifying precision data and by calculating also the upper verification limit (UVL).
Results
Matrix effect
When comparing plasma and serum samples, a statistical significant agreement and interchangeability between the two matrices was obtained, as shown in Figure 1 for TSH assay. In particular, the mean ± standard deviations (SD) for serum and plasma samples were equal to 5.88±16.63 and 5.78±15.96 (t-test for comparing groups, t=0.02 P=0.978), respectively.
Precision study
Table 1 reports the results for the precision study performed using the manufacturers IQC materials. The Repeatability and the within-lab precision were compared with those claimed by the manufacturer. The comparison was performed according to the EP15-A3 guideline and included the calculation of the UVL when necessary. Table 2 shows the precision study results obtained using the Biorad 1,2,3 Multiqual Unassay QC material.
Table 1
| Measurand | Level | Mean | Repeatability (CV%) | Within-lab precision (CV%) |
|---|---|---|---|---|
| TSH (mIU/L) | Level 1 | 0.667 | 3.52* | 3.52 |
| Level 2 | 39.67 | 2.50 | 4.09* | |
| FT3 (pmol/L) | Level 1 | 4.08 | 1.29 | 2.76 |
| Level 2 | 16.15 | 1.49 | 2.36 | |
| FT4 (pmol/L) | Level 1 | 10.26 | 3.62 | 3.95 |
| Level 2 | 38.56 | 0.84 | 1.33 | |
| Tg (µg/L) | Level 1 | 9.38 | 2.07 | 2.32 |
| Level 2 | 69.34 | 2.24 | 1.65 | |
| Anti TPO (IU/mL) | Level 1 | 4.31 | 3.21* | 5.20 |
| Level 2 | 89.28 | 3.86* | 4.78 | |
| Anti Tg (IU/mL) | Level 1 | 18.62 | 4.34 | 4.06 |
| Level 2 | 249.9 | 2.74 | 3.04 |
*, indicates that imprecision value was higher than that declared by manufacturers, also after the calculation of UVL as suggested by the CLSI EP15-A3. IQC, internal quality control; UVL, upper verification limit.
Table 2
| Measurand | Level | Mean | Repeatability (CV%) | Within-lab precision (CV%) |
|---|---|---|---|---|
| TSH (mIU/L) | Level 1 | 0.30 | 3.05 | 5.51 |
| Level 2 | 3.51 | 3.65 | 4.06 | |
| Level 3 | 22.96 | 2.29 | 4.20 | |
| FT3 (pmol/L) | Level 1 | 4.15 | 3.61 | 3.61 |
| Level 2 | 10.20 | 2.23 | 2.33 | |
| Level 3 | 16.53 | 1.39 | 2.11 | |
| FT4 (pmol/L) | Level 1 | 9.35 | 3.55 | 3.55 |
| Level 2 | 28.86 | 1.46 | 1.46 | |
| Level 3 | 44.65 | 1.50 | 1.50 | |
| Tg (ug/L) | Level 1 | 0.61 | 3.97 | 3.97 |
| Level 2 | 1.07 | 2.04 | 2.24 | |
| Level 3 | 1.55 | 3.51 | 3.51 |
Method comparison study and matrix effect evaluation
Results of the evaluation of comparability of TSH, FT3, FT4, Tg, TPOAb and TgAb, including the Passing-Bablok regressions and the Bland Altman analyses were reported in Table 3. The results for TSH and FT3 showed a good comparability between the Mindray CL-2000i and the Roche Cobas 6000 e601, with an absence of constant and proportional bias (Figure 2).
Table 3
| Measurand | Passing-Bablok regression results | Bland Altman results, Bias (95% CI) | |
|---|---|---|---|
| Slope (95% CI) | Intercept (95% CI) | ||
| TSH | 1.01 (0.98 to 1.04) | −0.13 (−0.19 to −0.09)# | −0.047 (−0.290 to 0.197) |
| FT3 | 0.93 (0.89 to 0.98)# | 0.27 (0.08 to 0.50)# | −0.037 (−0.136 to 0.062) |
| FT4 | 0.80 (0.77 to 0.84)# | −1.76 (−2.36 to −1.15)# | −4.830 (−5.123 to −4.537)# |
| Tg | 1.29 (1.25 to 1.33)# | −0.12 (−0.39 to 0.01) | 7.352 (4.221 to 10.484)# |
| TPOAb | 0.73 (0.57 to 0.82)# | −1.25 (−1.76 to −0.68)# | −13.519 (−23.686 to −3.356)# |
| TgAb | 0.07 (0.01 to 0.22)# | 4.12 (3.42 to 4.50)# | −53.391 (−68.378 to −38.395)# |
#, indicates a significant variation with respect to the expected results under the hypothesis of test equality. Bland Altman analyses were performed by calculating the differences between the Mindray CL2000i and the comparative method in the appropriate measurement units. Passing-Bablok regressions were obtained by calculating the results for the following equation: Mindray CL2000i = slope × comparative method + intercept.
Limits of detection (LOD) and linearity
The linearity test demonstrated a regression coefficient (R2) equivalent to 0.9996 for Tg and 0.9985 for TSH and Spearman’s correlation coefficient (R) was 1.000 (Figure 3 and Table 4) for both tests considered.
Table 4
| Test unit | Concentration range | Linear equation | r2 | R Spearman; P value |
|---|---|---|---|---|
| Tg (µg/L) | 0.115–235.68 | y=1.0037x+1.1054 | 0.9996 | R=1.000; P<0.0001 |
| TSH (mUI/L) | 1.5–96.33 | y=0.9915x−0.3258 | 0.9985 | R=1.000; P<0.0001 |
The low detection limits were found to be 0.0005 mUI/L (CV% =9.46) and 0.0936 (CV% =10.76) for TSH and Tg, respectively.
Discussion
In the last few years, a discussion has been raised on the reproducibility crisis in clinical research, generated from a lack of confidence of results among clinicians due to inappropriate statistical methods or poor experimental competences (9).
Thyroid dysfunction and treatment follow-up require accurate measurement of thyroid hormones. Most thyroid disease is treated on an outpatient basis; thus, assays have to be rapid and cost-effective for optimal patient care. Immunoassays are most commonly used because of their ease and availability, even if liquid chromatography-tandem mass spectrometry (LC-MS/MS) assays are available as reference methods. LC-MS/MS assays are, in fact, much more specific, but are laborious with a high machine cost. Projects for improving laboratory results of thyroid testing received, therefore, an increasingly attention in the last few years, particularly as regards TSH measurement, and some major concerns have been identified on the need to improve the harmonization/standardization of the tests results across all the analytical methods (10-12). The first step in this harmonization process is the accurate evaluation of the performance characteristics of the assays, particularly precision and trueness, and this was the aim of this study.
Precision has been differently defined, depending on the guidelines referenced. One definition based on a robust protocol is described in the CLSI EP15-A3. In this guideline, precision may be decomposed in its components, repeatability (also known as intra-day precision), between-days and within-lab (also known as total precision), even if only the first and the last have valuable interest. In this study, both repeatability and within-lab precisions were evaluated by using manufacturers’ IQC and third part quality controls (Bio-Rad Multiqual 1,2,3 Unassayed). The results obtained underlined that precision goals claimed by manufacturers were verifiable for most of the measurands, especially for the within-laboratory precision. Considering the results for the third-part quality controls, results were similar to manufacturer’s IQC, especially for within-laboratory precision; indeed, differences were more relevant for repeatability.
Other important aspects evaluated in this study were (I) the matrix effect; (II) methods comparability; (III) linearity; and (IV) LOD. Considering the first point, the obtained results showed that completely comparable results for serum and plasma, supporting the workable implementation of the Mindray CL-2000i system irrespectively of the sample matrix used by laboratories for thyroid testing. The method comparison evaluation calls for the notion that thyroid tests results, even when performed by different laboratories, should be reliably and successfully interpreted (4). The results obtained for TSH and FT3 showed an optimal comparability between the Mindray CL-2000i system and comparative method (Roche Cobas e601), with the absence of proportional biases. Differently, a slight (FT4 and Tg) to a moderate degree (TgAb) of proportional and/or constant bias was found for the other measurands and similar findings were already published in other studies comparing analytical performances of different analytical systems for thyroid testing (13,14). Finally, linearity evaluation underlined excellent results, while LOD evaluation demonstrates a better performance than the comparative method, being the Roche Cobas e601 LOD 0.005 mUI/L for TSH and the Beckman Coulter 0.1 ng/mL for Tg, respectively.
In summary, with respect to the current state-of-the-art of the thyroid testing by immunoassay systems, the results obtained in the present study support the comparability of the Mindray CL-2000i automated system with respect to other immunoassay methods for the thyroid testing. Additional features of Mindray CL-2000i were the good assays linearity and the low LOD obtainable for TSH and Tg.
Acknowledgments
Funding: None.
Footnote
Provenance and Peer Review: This article was commissioned by the editorial office, Journal of Laboratory and Precision Medicine for the series “Laboratory Medicine-25 Years on”. The article has undergone external peer review.
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jlpm.2018.10.03). The series “Laboratory Medicine-25 Years on” was commissioned by the editorial office without any funding or sponsorship. Mario Plebani served as an unpaid Guest Editor of the series. The authors have no other conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The institutional ethical approval and individual informed consent were waived.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- LeFevre MLU.S. Preventive Services Task Force. Screening for thyroid dysfunction: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2015;162:641-50. [Crossref] [PubMed]
- Vanderpump MP. The epidemiology of thyroid disease. Br Med Bull 2011;99:39-51. [Crossref] [PubMed]
- Sarkar R. TSH Comparison Between Chemiluminescence (Architect) and Electrochemiluminescence (Cobas) Immunoassays: An Indian Population Perspective. Indian J Clin Biochem 2014;29:189-95. [Crossref] [PubMed]
- Tozzoli R, Bizzaro N. Harmonization in autoimmune thyroid disease diagnostics. Clin Chem Lab Med 2018;56:1778-82. [PubMed]
- Antonelli G, Padoan A, Aita A, et al. Verification of examination procedures in clinical laboratory for imprecision, trueness and diagnostic accuracy according to ISO 15189:2012: a pragmatic approach. Clin Chem Lab Med 2017;55:1501-8. [PubMed]
- ISO 15189. Medical Laboratories -- Requirements for quality and competence. Geneva, International Organization for Standardization (ISO), 2012.
- Piketty ML, Polak M, Flechtner I, et al. False biochemical diagnosis of hyperthyroidism in streptavidin-biotin-based immunoassays: the problem of biotin intake and related interferences. Clin Chem Lab Med 2017;55:780-8. [Crossref] [PubMed]
- Clinical and Laboratory Standards Institute (CLSI). User Verification of Precision and Estimation of Bias; Approved Guideline - Third Edition. 2014. CLSI EP15-A3, Wayne, PA, USA.
- Ioannidis JPA. The Reproducibility Wars: Successful, Unsuccessful, Uninterpretable, Exact, Conceptual, Triangulated, Contested Replication. Clin Chem 2017;63:943-5. [Crossref] [PubMed]
- Plebani M. Harmonization in laboratory medicine: the complete picture. Clin Chem Lab Med 2013;51:741-51. [Crossref] [PubMed]
- Clerico A, Ripoli A, Fortunato A, et al. Harmonization protocols for TSH immunoassays: a multicenter study in Italy. Clin Chem Lab Med 2017;55:1722-33. [Crossref] [PubMed]
- Plebani M. Towards a new paradigm in laboratory medicine: the five rights. Clin Chem Lab Med 2016;54:1881-91. [Crossref] [PubMed]
- Fillée C, Cumps J, Ketelslegers JM. Comparison of three free T4 (FT4) and free T3 (FT3) immunoassays in healthy subjects and patients with thyroid diseases and severe non-thyroidal illnesses. Clin Lab 2012;58:725-36. [PubMed]
- Diana T, Wüster C, Olivo PD, et al. Performance and Specificity of 6 Immunoassays for TSH Receptor Antibodies: A Multicenter Study. Eur Thyroid J 2017;6:243-9. [Crossref] [PubMed]
Cite this article as: Padoan A, Cosma C, Plebani M. Evaluation of the analytical performances of six measurands for thyroid functions of Mindray CL-2000i system. J Lab Precis Med 2018;3:93.
