Cited 0 times in Scipus Cited Count

Towards global model generalizability: independent cross-site feature evaluation for patient-level risk prediction models using the OHDSI network

Authors
Naderalvojoud, B | Curtin, CM | Yanover, C | El-Hay, T | Choi, B | Park, RW  | Tabuenca, JG | Reeve, MP | Falconer, T | Humphreys, K | Asch, SM | Hernandez-Boussard, T
Citation
Journal of the American Medical Informatics Association : JAMIA, 31(5). : 1051-1061, 2024
Journal Title
Journal of the American Medical Informatics Association : JAMIA
ISSN
1067-50271527-974X
Abstract
Background: Predictive models show promise in healthcare, but their successful deployment is challenging due to limited generalizability. Current external validation often focuses on model performance with restricted feature use from the original training data, lacking insights into their suitability at external sites. Our study introduces an innovative methodology for evaluating features during both the development phase and the validation, focusing on creating and validating predictive models for post-surgery patient outcomes with improved generalizability. Methods: Electronic health records (EHRs) from 4 countries (United States, United Kingdom, Finland, and Korea) were mapped to the OMOP Common Data Model (CDM), 2008-2019. Machine learning (ML) models were developed to predict post-surgery prolonged opioid use (POU) risks using data collected 6 months before surgery. Both local and cross-site feature selection methods were applied in the development and external validation datasets. Models were developed using Observational Health Data Sciences and Informatics (OHDSI) tools and validated on separate patient cohorts. Results: Model development included 41 929 patients, 14.6% with POU. The external validation included 31 932 (UK), 23 100 (US), 7295 (Korea), and 3934 (Finland) patients with POU of 44.2%, 22.0%, 15.8%, and 21.8%, respectively. The top-performing model, Lasso logistic regression, achieved an area under the receiver operating characteristic curve (AUROC) of 0.75 during local validation and 0.69 (SD ¼ 0.02) (averaged) in external validation. Models trained with cross-site feature selection significantly outperformed those using only features from the development site through external validation (P < .05). Conclusions: Using EHRs across four countries mapped to the OMOP CDM, we developed generalizable predictive models for POU. Our approach demonstrates the significant impact of cross-site feature selection in improving model performance, underscoring the importance of incorporating diverse feature sets from various clinical settings to enhance the generalizability and utility of predictive healthcare models.
Keywords

MeSH

DOI
10.1093/jamia/ocae028
PMID
38412331
Appears in Collections:
Journal Papers > School of Medicine / Graduate School of Medicine > Biomedical Informatics
Ajou Authors
박, 래웅
Full Text Link
Files in This Item:
There are no files associated with this item.
Export

qrcode

해당 아이템을 이메일로 공유하기 원하시면 인증을 거치시기 바랍니다.

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Browse