TY - JOUR
T1 - Machine learning–based mortality prediction models using national liver transplantation registries are feasible but have limited utility across countries
AU - Ivanics, Tommy
AU - So, Delvin
AU - Claasen, Marco P.A.W.
AU - Wallace, David
AU - Patel, Madhukar S.
AU - Gravely, Annabel
AU - Choi, Woo Jin
AU - Shwaartz, Chaya
AU - Walker, Kate
AU - Erdman, Lauren
AU - Sapisochin, Gonzalo
N1 - Publisher Copyright:
© 2022 American Society of Transplantation & American Society of Transplant Surgeons
PY - 2023/1
Y1 - 2023/1
N2 - Many countries curate national registries of liver transplant (LT) data. These registries are often used to generate predictive models; however, potential performance and transferability of these models remain unclear. We used data from 3 national registries and developed machine learning algorithm (MLA)–based models to predict 90-day post-LT mortality within and across countries. Predictive performance and external validity of each model were assessed. Prospectively collected data of adult patients (aged ≥18 years) who underwent primary LTs between January 2008 and December 2018 from the Canadian Organ Replacement Registry (Canada), National Health Service Blood and Transplantation (United Kingdom), and United Network for Organ Sharing (United States) were used to develop MLA models to predict 90-day post-LT mortality. Models were developed using each registry individually (based on variables inherent to the individual databases) and using all 3 registries combined (variables in common between the registries [harmonized]). The model performance was evaluated using area under the receiver operating characteristic (AUROC) curve. The number of patients included was as follows: Canada, n = 1214; the United Kingdom, n = 5287; and the United States, n = 59,558. The best performing MLA-based model was ridge regression across both individual registries and harmonized data sets. Model performance diminished from individualized to the harmonized registries, especially in Canada (individualized ridge: AUROC, 0.74; range, 0.73-0.74; harmonized: AUROC, 0.68; range, 0.50-0.73) and US (individualized ridge: AUROC, 0.71; range, 0.70-0.71; harmonized: AUROC, 0.66; range, 0.66-0.66) data sets. External model performance across countries was poor overall. MLA-based models yield a fair discriminatory potential when used within individual databases. However, the external validity of these models is poor when applied across countries. Standardization of registry-based variables could facilitate the added value of MLA-based models in informing decision making in future LTs.
AB - Many countries curate national registries of liver transplant (LT) data. These registries are often used to generate predictive models; however, potential performance and transferability of these models remain unclear. We used data from 3 national registries and developed machine learning algorithm (MLA)–based models to predict 90-day post-LT mortality within and across countries. Predictive performance and external validity of each model were assessed. Prospectively collected data of adult patients (aged ≥18 years) who underwent primary LTs between January 2008 and December 2018 from the Canadian Organ Replacement Registry (Canada), National Health Service Blood and Transplantation (United Kingdom), and United Network for Organ Sharing (United States) were used to develop MLA models to predict 90-day post-LT mortality. Models were developed using each registry individually (based on variables inherent to the individual databases) and using all 3 registries combined (variables in common between the registries [harmonized]). The model performance was evaluated using area under the receiver operating characteristic (AUROC) curve. The number of patients included was as follows: Canada, n = 1214; the United Kingdom, n = 5287; and the United States, n = 59,558. The best performing MLA-based model was ridge regression across both individual registries and harmonized data sets. Model performance diminished from individualized to the harmonized registries, especially in Canada (individualized ridge: AUROC, 0.74; range, 0.73-0.74; harmonized: AUROC, 0.68; range, 0.50-0.73) and US (individualized ridge: AUROC, 0.71; range, 0.70-0.71; harmonized: AUROC, 0.66; range, 0.66-0.66) data sets. External model performance across countries was poor overall. MLA-based models yield a fair discriminatory potential when used within individual databases. However, the external validity of these models is poor when applied across countries. Standardization of registry-based variables could facilitate the added value of MLA-based models in informing decision making in future LTs.
KW - international liver registry
KW - liver transplantation
KW - machine learning algorithm
KW - outcome prediction
UR - http://www.scopus.com/inward/record.url?scp=85146848186&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85146848186&partnerID=8YFLogxK
U2 - 10.1016/j.ajt.2022.12.002
DO - 10.1016/j.ajt.2022.12.002
M3 - Article
C2 - 36695623
AN - SCOPUS:85146848186
SN - 1600-6135
VL - 23
SP - 64
EP - 71
JO - American Journal of Transplantation
JF - American Journal of Transplantation
IS - 1
ER -