Medizinische Universität Graz Austria/Österreich - Forschungsportal - Medical University of Graz
Gewählte Publikation:
Jauk, S.
Performance and User Acceptance of a Machine Learning-Based Delirium Risk Stratification Tool in Clinical Routine
PhD-Studium (Doctor of Philosophy); Humanmedizin; [ Dissertation ] Medical University of Graz; 2021. pp. 139
[OPEN ACCESS]
FullText
- Autor*innen der Med Uni Graz:
- Betreuer*innen:
- Avian Alexander
- Berghold Andrea
- Schulz Stefan
- Altmetrics:
- Abstract:
- In clinical routine, early identification of patients with life-threatening risks is crucial in order to initiate preventive actions as quickly as possible. Clinical prediction models stratify patients according to their risk and thus support healthcare professionals in their decision-making. Owing to the increasing amount of clinical data stored in electronic health record (EHR) systems, numerous machine learning-based prediction models have been developed over the last years. A main advantage of combining machine learning and EHR data is that no additional information needs to be assessed, which saves resources and allows for routine risk stratification in hospitals. Despite demonstrations of outstanding prognostic performance in test data sets, only few machine learning models have been implemented in clinical settings. Therefore, little is known about their clinical performance and their acceptance by clinicians. The goal of this thesis was to evaluate a machine learning-based risk stratification tool in clinical routine. The predicted outcome of the tool is delirium, a syndrome of acute confusional state with high morbidity and mortality in hospitalised patients. The evaluation addressed three aspects: (1) the prospective performance of the delirium risk stratification algorithm in a seven-months pilot study; (2) the technology acceptance of the tool by healthcare professionals; and (3) the long-term performance when implemented in five hospitals across the Austrian region of Styria. The results demonstrate that the algorithm achieved a stable performance for internal medicine and surgical patients in clinical routine during a pilot study and in the long term. Delirium risk predictions by the algorithm were in high agreement with risk ratings by clinical experts for a sample of general internal medicine and gastroenterology patients. Overall, healthcare professionals rated the usefulness, ease of use and output quality positively and appreciated the automatic and fast prediction. However, the reported use of the tool was still low and more promotion and training sessions will be needed in future deployments. The evaluation also revealed weaknesses of the machine learning-based tool, e.g. a decrease in performance when applied to a cardiology department with a more complex patient cohort. In addition, a low observed incidence of delirium in the EHR data limited the evaluation, but ways to overcome this limitation in future are discussed. To conclude, this thesis provides new insights into the clinical performance of a machine learning-based risk stratification tool and illustrates its strengths and weaknesses. It demonstrates the high predictive performance of machine learning-based delirium prediction and the positive acceptance by healthcare professionals. Even though the focus of this thesis was the prediction of delirium, the results will support the evaluation and critical appraisal of machine learning models for different clinical outcomes in future.