1
|
Horvath AR, Bell KJL, Ceriotti F, Jones GRD, Loh TP, Lord S, Sandberg S. Outcome-based analytical performance specifications: current status and future challenges. Clin Chem Lab Med 2024; 62:1474-1482. [PMID: 38836433 DOI: 10.1515/cclm-2024-0125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Accepted: 05/18/2024] [Indexed: 06/06/2024]
Abstract
Analytical performance specifications (APS) based on outcomes refer to how 'good' the analytical performance of a test needs to be to do more good than harm to the patient. Analytical performance of a measurand affects its clinical performance. Without first setting clinical performance requirements, it is difficult to define how good analytically the test needs to be to meet medical needs. As testing is indirectly linked to health outcomes through clinical decisions on patient management, often simulation-based studies are used to assess the impact of analytical performance on the probability of clinical outcomes which is then translated to Model 1b APS according to the Milan consensus. This paper discusses the related key definitions, concepts and considerations that should assist in finding the most appropriate methods for deriving Model 1b APS. We review the advantages and limitations of published methods and discuss the criteria for transferability of Model 1b APS to different settings. We consider that the definition of the clinically acceptable misclassification rate is central to Model 1b APS. We provide some examples and guidance on a more systematic approach for first defining the clinical performance requirements for tests and we also highlight a few ideas to tackle the future challenges associated with providing outcome-based APS for laboratory testing.
Collapse
Affiliation(s)
- Andrea Rita Horvath
- Department of Chemical Pathology, 441551 New South Wales Health Pathology, Prince of Wales Hospital , Sydney, Australia
- School of Public Health, University of Sydney, Sydney, Australia
- Faculty of Medicine, University of New South Wales, Sydney, Australia
| | - Katy J L Bell
- Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
| | - Ferruccio Ceriotti
- Clinical Laboratory, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - Graham R D Jones
- Faculty of Medicine, University of New South Wales, Sydney, Australia
- Department of Chemical Pathology, SydPath, St Vincent's Hospital, Darlinghurst, NSW, Australia
| | - Tze Ping Loh
- Department of Laboratory Medicine, National University Hospital, Singapore, Singapore
| | - Sally Lord
- School of Medicine, University of Notre Dame, Darlinghurst, New South Wales, Australia
- NHMRC Clinical Trials Centre, The University of Sydney, Camperdown, New South Wales, Australia
| | - Sverre Sandberg
- Norwegian Organization for Quality Improvement of Laboratory Examinations (NOKLUS), Haraldsplass Deaconess Hospital, Bergen, Norway
- Norwegian Porphyria Centre, Department of Medical Biochemistry and Pharmacology, Haukeland University Hospital, Bergen, Norway
- Institute of Public Health and Primary Health Care, University of Bergen, Bergen, Norway
| |
Collapse
|
2
|
Zayed AM, Saegeman V, Delvaux N. Establishing the Reportable Interval for Routine Clinical Laboratory Tests: A Data-Driven Strategy Leveraging Retrospective Electronic Medical Record Data. J Appl Lab Med 2024; 9:776-788. [PMID: 38642405 DOI: 10.1093/jalm/jfae021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 02/05/2024] [Indexed: 04/22/2024]
Abstract
BACKGROUND This paper presents a data-driven strategy for establishing the reportable interval in clinical laboratory testing. The reportable interval defines the range of laboratory result values beyond which reporting should be withheld. The lack of clear guidelines and methodology for determining the reportable interval has led to potential errors in reporting and patient risk. METHODS To address this gap, the study developed an integrated strategy that combines statistical analysis, expert review, and hypothetical outlier calculations. A large data set from an accredited clinical laboratory was utilized, analyzing over 124 million laboratory test records from 916 distinct tests. The Dixon test was applied to identify outliers and establish the highest and lowest non-outlier result values for each test, which were validated by clinical pathology experts. The methodology also included matching the reportable intervals with relevant Logical Observation Identifiers Names and Codes (LOINC) and Unified Code for Units of Measure (UCUM)-valid units for broader applicability. RESULTS Upon establishing the reportable interval for 135 routine laboratory tests (493 LOINC codes), we applied these to a primary care laboratory data set of 23 million records, demonstrating their efficacy with over 1% of result records identified as implausible. CONCLUSIONS We developed and tested a data-driven strategy for establishing reportable intervals utilizing large electronic medical record (EMR) data sets. Implementing the established interval in clinical laboratory settings can improve autoverification systems, enhance data reliability, and reduce errors in patient care. Ongoing refinement and reporting of cases exceeding the reportable limits will contribute to continuous improvement in laboratory result management and patient safety.
Collapse
Affiliation(s)
- Ahmed M Zayed
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
- Laboratory Medicine Department, Menoufia University National Liver Institute, Shebin El-Kom, Egypt
| | - Veroniek Saegeman
- Vitaz Sint-Niklaas Moerland, Clinical Laboratory, Sint-Niklaas, Oost-Vlaanderen, Belgium
| | - Nicolas Delvaux
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
3
|
Kang T, Yoo J, Jekarl DW, Chae H, Kim M, Park YJ, Oh EJ, Kim Y. Indirect Method for Estimation of Reference Intervals of Inflammatory Markers. Ann Lab Med 2023; 43:55-63. [PMID: 36045057 PMCID: PMC9467833 DOI: 10.3343/alm.2023.43.1.55] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 05/09/2022] [Accepted: 08/17/2022] [Indexed: 12/27/2022] Open
Abstract
Background The direct method for reference interval (RI) estimating is limited due to the requirement of resources, difficulties in defining a non-diseased population, or ethical problems in obtaining samples. We estimated the RI for inflammatory biomarkers using an indirect method (RII). Methods C-reactive protein (CRP), erythrocyte sedimentation rate (ESR) and presepsin (PSEP) data of patients visiting a single hospital were retrieved from April 2009 to April 2021. Right-skewed data were transformed using the Box-Cox transformation method. A mixed population of non-diseased and diseased distributions was assumed, followed by latent profile analysis for the two classes. The intersection point of the distribution curve was estimated as the RI. The influence of measurement size was evaluated as the ratio of abnormal values and adjustment (n×bandwidth) of the distribution curve. Results The RIs estimated by the proposed RII method (existing method) were as follows: CRP, 0-4.1 (0-4.7) mg/L; ESR, 0-10.2 (0-15) mm/hr and PSEP, 0-411 (0-300) pg/mL. Measurement sizes ≥2,500 showed stable results. An abnormal-to-normal value ratio of 0.5 showed the most accurate result for CRP. Adjustment values ≤5 or >5 were applicable for a measurement size <25,000 or ≥25,000, respectively. Conclusions The proposed RII method could provide additional information for RI verification or estimation with some limitations.
Collapse
Affiliation(s)
- Taewon Kang
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Jeaeun Yoo
- Department of Laboratory Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Dong Wook Jekarl
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea.,Research and Development Institute for In Vitro Diagnostic Medical Devices, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Hyojin Chae
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Myungshin Kim
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Yeon-Joon Park
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Eun-Jee Oh
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea.,Research and Development Institute for In Vitro Diagnostic Medical Devices, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Yonggoo Kim
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| |
Collapse
|
4
|
Broecker-Preuss M, Arzideh F, Best J, Canbay A, Özçürümez M, Manka P. Comparison of age- and sex-dependent reference limits derived from distinct sources for metabolic measurands in basic liver diagnostics. ZEITSCHRIFT FUR GASTROENTEROLOGIE 2023; 61:50-59. [PMID: 36623543 DOI: 10.1055/a-1994-0809] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
BACKGROUND Reference intervals for basic liver laboratory diagnostic rely on manufacturers' information, remaining unchanged for more than 20 years. This ignores known age and sex dependencies. METHODS We performed a retrospective cross-sectional study to compare the age-dependent distribution of flagged and non-flagged laboratory findings between reference limits from 3 distinct sources: manufacturer, published reference study, and the truncated maximum likelihood method applied on a cohort of inpatients aged 18-100 years. Discordance rates adjusted for the permissible analytical uncertainty are reported for serum levels of albumin (n= 150,550), alkaline phosphatase (n= 433,721), gamma-GT (n=580,012), AST (n= 510,620), and ALT (n= 704,546). RESULTS The number of flagged findings differed notably between reference intervals compared, except for alkaline phosphatase. AST and alkaline phosphatase increased with age in women. Overall discordance for AP, AST, and ALT remained below 10%, respectively, in both sexes. Albumin decreased with age which led to discordant flags in up to 22% in patients ≥70 years. GGT and ALT peaked in 50-59-year-old men with up to 23.5% and 22.8% discordant flags, respectively. CONCLUSION We assessed the impact of different reference limits on liver related laboratory results and found up to 25 % discordant flags. We suggest to further analyse the diagnostic and economic effects of reference limits adapted to the population of interest even for well-established basic liver diagnostics.
Collapse
Affiliation(s)
- Martina Broecker-Preuss
- Medizinische Klinik, Universitätsklinikum Knappschaftskrankenhaus Bochum, Ruhr-Universität Bochum, Bochum, Germany
| | - Farhad Arzideh
- Medizinische Klinik, Universitätsklinikum Knappschaftskrankenhaus Bochum, Ruhr-Universität Bochum, Bochum, Germany
| | - Jan Best
- Medizinische Klinik, Universitätsklinikum Knappschaftskrankenhaus Bochum, Ruhr-Universität Bochum, Bochum, Germany
| | - Ali Canbay
- Medizinische Klinik, Universitätsklinikum Knappschaftskrankenhaus Bochum, Ruhr-Universität Bochum, Bochum, Germany
| | - Mustafa Özçürümez
- Medizinische Klinik, Universitätsklinikum Knappschaftskrankenhaus Bochum, Ruhr-Universität Bochum, Bochum, Germany
| | - Paul Manka
- Medizinische Klinik, Universitätsklinikum Knappschaftskrankenhaus Bochum, Ruhr-Universität Bochum, Bochum, Germany
| |
Collapse
|