Turning data into decisions : clinical decision support in orthopaedic oncology

Abstract

Background: The treatment of patients with skeletal metastases is predicated on each patient’s estimated survival. In order to maximize function and quality of life, orthopaedic surgeons must carefully avoid over- or undertreatment of the disease. Unfortunately, physician estimates are notoriously inaccurate and there are no validated means by which to estimate patient survival in patients with long-bone skeletal metastases. The purpose of this thesis is to apply machine learning (ML) approaches to (1) develop a clinical decision support (CDS) tool capable of estimating survival in patients with operable skeletal metastases, and (2) establish guidelines so that this approach may be used in other relevant topics within the field of orthopaedics.

Methods: We first defined the scope of the problem using data from the Karolinska Skeletal Metastasis Registry. We then developed objective criteria by which to estimate patient survival using data gleaned from the Memorial Sloan-Kettering Skeletal Metastasis Database (n=189). We employed ML techniques to find patterns within the data associated with short- and long-term survival. We chose three and 12 months because they are widely accepted to guide orthopaedic surgical decisionmaking. We developed an Artificial Neural Network (ANN), a Bayesian Belief Network (BBN), and a traditional Logistic Regression (LR) model. Each resulting model was internally validated and compared using Receiver Operator Characteristic (ROC) analysis. In addition, we performed decision analysis to determine which model, if any, was suited for clinical use. Next, we externally validated the models using Scandinavian Registry data (n=815), and again using data collected by the Societ. Italiana di Ortopedia e Traumatologia (SIOT) (n=287). We then created a web-based CDS tool as well as the infrastructure to collect prospective data on a global scale, so the models could be improved over time. Finally, we used BBN modeling to describe the hierarchical relationships between features associated with the treatment of highgrade soft tissue sarcomas (STS), and codify this complex information into a graphical representation to promote a more thorough understanding of the disease process.

Results: We found that implant failures in patients with skeletal metastases remain relatively common—even in the revision setting—as patients outlive their implants. On the other hand, perioperative deaths are relatively common, indicating that an estimation of life expectancy should be part of the surgical decision making process. Using ML approaches, we found several criteria that can be used to estimate longevity in this patient population. When compared to other techniques, the ANN model was most accurate, and also resulted in highest net benefit on decision analysis, compared to the BBN and LR models. However, the BBN is the best suited to accommodate missing data, which is common in the clinical setting. The three- and 12-month BBN models were successfully externally validated using the SSMR database (Area under the ROC curve (AUC) of 0.79 and 0.76, respectively), and again using SIOT data (AUC 0.80 and 0.77). In the setting of high-grade, completely excised STS, BBN Modeling identified the first-degree associates of disease-specific survival to be the size of the primary tumor, and the presence and timing of local and distant recurrence.

Conclusions: We successfully developed and validated a CDS tool designed to estimate survival in patients with operable skeletal metastases. In addition, we made this tool available to orthopaedic surgeons, worldwide, at www.pathfx.org. We also created an international skeletal metastasis registry to continue to collect data on patients with skeletal metastases. Within this framework, prognostic models have the capacity to improve over time, as treatment philosophies evolve and more effective systemic therapies become available. These techniques may now be applied to other disciplines, in an effort to turn quality data into decision support tools.