1
|
Saffari SE, Ning Y, Xie F, Chakraborty B, Volovici V, Vaughan R, Ong MEH, Liu N. AutoScore-Ordinal: an interpretable machine learning framework for generating scoring models for ordinal outcomes. BMC Med Res Methodol 2022; 22:286. [PMID: 36333672 PMCID: PMC9636613 DOI: 10.1186/s12874-022-01770-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 10/25/2022] [Indexed: 11/06/2022] Open
Abstract
Background Risk prediction models are useful tools in clinical decision-making which help with risk stratification and resource allocations and may lead to a better health care for patients. AutoScore is a machine learning–based automatic clinical score generator for binary outcomes. This study aims to expand the AutoScore framework to provide a tool for interpretable risk prediction for ordinal outcomes. Methods The AutoScore-Ordinal framework is generated using the same 6 modules of the original AutoScore algorithm including variable ranking, variable transformation, score derivation (from proportional odds models), model selection, score fine-tuning, and model evaluation. To illustrate the AutoScore-Ordinal performance, the method was conducted on electronic health records data from the emergency department at Singapore General Hospital over 2008 to 2017. The model was trained on 70% of the data, validated on 10% and tested on the remaining 20%. Results This study included 445,989 inpatient cases, where the distribution of the ordinal outcome was 80.7% alive without 30-day readmission, 12.5% alive with 30-day readmission, and 6.8% died inpatient or by day 30 post discharge. Two point-based risk prediction models were developed using two sets of 8 predictor variables identified by the flexible variable selection procedure. The two models indicated reasonably good performance measured by mean area under the receiver operating characteristic curve (0.758 and 0.793) and generalized c-index (0.737 and 0.760), which were comparable to alternative models. Conclusion AutoScore-Ordinal provides an automated and easy-to-use framework for development and validation of risk prediction models for ordinal outcomes, which can systematically identify potential predictors from high-dimensional data.
Collapse
|
2
|
Mahony NO, Campbell S, Krpalkova L, Carvalho A, Walsh J, Riordan D. Representation Learning for Fine-Grained Change Detection. SENSORS (BASEL, SWITZERLAND) 2021; 21:4486. [PMID: 34209075 PMCID: PMC8271830 DOI: 10.3390/s21134486] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Revised: 06/16/2021] [Accepted: 06/25/2021] [Indexed: 11/16/2022]
Abstract
Fine-grained change detection in sensor data is very challenging for artificial intelligence though it is critically important in practice. It is the process of identifying differences in the state of an object or phenomenon where the differences are class-specific and are difficult to generalise. As a result, many recent technologies that leverage big data and deep learning struggle with this task. This review focuses on the state-of-the-art methods, applications, and challenges of representation learning for fine-grained change detection. Our research focuses on methods of harnessing the latent metric space of representation learning techniques as an interim output for hybrid human-machine intelligence. We review methods for transforming and projecting embedding space such that significant changes can be communicated more effectively and a more comprehensive interpretation of underlying relationships in sensor data is facilitated. We conduct this research in our work towards developing a method for aligning the axes of latent embedding space with meaningful real-world metrics so that the reasoning behind the detection of change in relation to past observations may be revealed and adjusted. This is an important topic in many fields concerned with producing more meaningful and explainable outputs from deep learning and also for providing means for knowledge injection and model calibration in order to maintain user confidence.
Collapse
Affiliation(s)
- Niall O’ Mahony
- Lero—The Irish Software Research Centre, V92 CX88 Tralee, Ireland; (S.C.); (L.K.); (A.C.); (J.W.); (D.R.)
- Department of Agricultural and Manufacturing Engineering, School of Science Technology Engineering and Maths (STEM), Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
- IMaR Research Centre, Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
| | - Sean Campbell
- Lero—The Irish Software Research Centre, V92 CX88 Tralee, Ireland; (S.C.); (L.K.); (A.C.); (J.W.); (D.R.)
- Department of Agricultural and Manufacturing Engineering, School of Science Technology Engineering and Maths (STEM), Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
- IMaR Research Centre, Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
| | - Lenka Krpalkova
- Lero—The Irish Software Research Centre, V92 CX88 Tralee, Ireland; (S.C.); (L.K.); (A.C.); (J.W.); (D.R.)
- Department of Agricultural and Manufacturing Engineering, School of Science Technology Engineering and Maths (STEM), Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
- IMaR Research Centre, Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
| | - Anderson Carvalho
- Lero—The Irish Software Research Centre, V92 CX88 Tralee, Ireland; (S.C.); (L.K.); (A.C.); (J.W.); (D.R.)
- Department of Agricultural and Manufacturing Engineering, School of Science Technology Engineering and Maths (STEM), Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
- IMaR Research Centre, Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
| | - Joseph Walsh
- Lero—The Irish Software Research Centre, V92 CX88 Tralee, Ireland; (S.C.); (L.K.); (A.C.); (J.W.); (D.R.)
- Department of Agricultural and Manufacturing Engineering, School of Science Technology Engineering and Maths (STEM), Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
- IMaR Research Centre, Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
| | - Daniel Riordan
- Lero—The Irish Software Research Centre, V92 CX88 Tralee, Ireland; (S.C.); (L.K.); (A.C.); (J.W.); (D.R.)
- Department of Agricultural and Manufacturing Engineering, School of Science Technology Engineering and Maths (STEM), Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
- IMaR Research Centre, Kerry Campus, Munster Technological University, V92 CX88 Tralee, Ireland
| |
Collapse
|