Journal Article Epublish
PubMed: 41281704 DOI: 10.1177/17562864251391095
External validation of a multiple sclerosis treatment decision score using data from the ProVal-MS cohort study.
Ther Adv Neurol Disord 18:17562864251391095
Authors: Stefan Buchka, Alexander Hapfelmeier, Jan S Kirschke, Verena Steuerwald, Markus Naumann, Iñaki Soto-Rey, Sven Olaf Rohr, Frank Kramer, Lars Behrens, Eva Oswald, Tania Kümpfel, Hanna Zimmermann, Verena S Hoffmann, Marlien Hagedorn, Fady Albashiti, Markus Krumbholz, Ulf Ziemann, Oliver Kohlbacher, Benjamin Sailer, Viola Braunmüller, Stephanie Biergans, Marius de Arruda Botelho Herr, Ulrike Ernemann, Eva Bürkle, Benjamin Bender, Andreas Daul, Christer Ruff, Jörg Römhild, Benedikt Wiestler, Dominik Sepp, Helmut Spengler, Peter Pallaoro, Martin Boeker, Florian Kohlmayer, Vera Dehmelt, Achim Berthele, Mark Mühlau, Paula Uibel, Josephine Wauschkuhn, Klaus Kuhn, Makbule Senel, Ioannis Vardakas, Daniela Taranu, Hans Armin Kestler, Nico Sollmann, Begüm Irmak Ön, Sandra Bilger, Ulrich Mansmann, Antonios Bayas, Joachim Havla … and 3 other authors
Affiliations
- The Institute for Medical Information Processing, Biometry, and Epidemiology, Medical Faculty, University of Munich, Munich, Germany.
- Technical University of Munich, TUM School of Medicine and Health, Institute for AI and Informatics in Medicine (AIIM), TUM University Hospital, Munich, Germany.
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, TUM Klinikum, Technical University of Munich, Munich, Germany.
- Department of Neurology and Clinical Neurophysiology, Medical Faculty, University of Augsburg, Augsburg, Germany.
- Digital Medicine, University Hospital of Augsburg, Augsburg, Germany.
- IT-Infrastructure for Translational Medical Research, University of Augsburg, Augsburg, Germany.
- Diagnostic and Interventional Neuroradiology, Faculty of Medicine, University of Augsburg, Augsburg, Germany.
- Institute of Clinical Neuroimmunology, University Hospital, LMU Munich, Munich, Germany.
- Institute of Diagnostic and Interventional Neuroradiology, LMU University Hospital, LMU Munich, Munich, Germany.
- Medical Data Integration Center, University Hospital, LMU Munich, Munich, Germany.
- Neu-Ulm University of Applied Sciences, Neu-Ulm, Germany.
- Department of Neurology and Stroke, and Hertie-Institute for Clinical Brain Research, Eberhard-Karls University of Tübingen, Tübingen, Germany.
- Medical Data Integration Center (meDIC), University Hospital Tübingen, Tübingen, Germany.
- Department of Computer Science, University of Tübingen, Tübingen, Germany.
- Institute for Bioinformatics and Medical Informatics, University of Tübingen, Tübingen, Germany.
- Institute for Translational Bioinformatics, University Hospital Tübingen, Tübingen, Germany.
- Department of Diagnostic and Interventional Neuroradiology, University Hospital Tübingen, Tübingen, Germany.
- University Hospital Tübingen, Tübingen, Germany.
- Technical University of Munich, TUM School of Medicine and Health, Data Integration Center, TUM University Hospital, Munich, Germany.
- Bitcare GmbH, Munich, Germany.
- Department of Neurology, School of Medicine, Technical University of Munich, Klinikum rechts der Isar, Munich, Germany.
- Department of Neurology, University Hospital Ulm, Ulm, Germany.
- Institute of Medical Systems Biology, Ulm University, Ulm, Germany.
- Department of Diagnostic and Interventional Radiology, University Hospital Ulm, Ulm, Germany.
- Staburo GmbH, Munich, Germany.
- MeDIC Ulm and Institute of Medical Systems Biology, University of Ulm, Ulm, Germany.
Abstract
BACKGROUND
The course of relapsing-remitting multiple sclerosis (RRMS), frequently preceded by the clinically isolated syndrome (CIS), is variable and challenging to predict. Given many treatment options available, prognostic algorithms are gaining importance in informing initial treatment decisions. However, to date, only a few externally validated exists. External validation, which involves the application of a model to independent data, is essential. Privacy-preserving federated analyses of individual-level data facilitate external validation using clinical datasets that are typically difficult to access.
OBJECTIVES
Using data from the ProVal-MS study to externally validate the multiple sclerosis treatment decision score (MS-TDS), a predictive algorithm for early RRMS and CIS. The MS-TDS predicts the probability of the occurrence of at least one new or enlarging T2 lesion within 6-24 months following the onset of the disease and supports choosing between initiating platform treatment or a 'wait-and-see' approach. A secondary objective is to demonstrate the feasibility of privacy-preserving federated concepts within the Data Integration for Future Medicine (DIFUTURE) consortium.
DESIGN
Prospective, multicentric, non-interventional cohort study (ProVal-MS) within DIFUTURE.
METHODS
The calibrated MS-TDS was evaluated using the area under the receiver operating characteristic curve (AUROC) and the Brier score in both pooled and distributed settings. A decision curve analysis (DCA) was used to evaluate the net benefit of treatment decisions made by the MS-TDS in comparison to those made by treating neurologists.
RESULTS
Of the 271 individuals diagnosed with CIS or early RRMS, 202 (78.2%) received platform treatment, while 59 (21.8%) did not receive treatment. The AUROC was 0.561 (95% CI: 0.492-0.630) in the pooled analysis and 0.567 (95% CI: 0.496-0.634) in the distributed analysis. DCA demonstrated a net benefit that was commensurate with that achieved by decisions made by experienced neurologists.
CONCLUSION
The external validation of the MS-TDS demonstrated low, non-significant predictive performance; however, it may serve as a useful complement, particularly for less-experienced neurologists. The distributed validation was found to be both feasible and compliant with data protection regulations.
MeSH
Sclerosis, Multiple Sclerosis, Cohort Studies, Algorithms, Patient Preference, Therapeutics, Neurologists, Prognosis