Jinkai Dong,1,* Minjie Duan,2,3,* Xiaozhu Liu,4,* Huan Li,5 Yang Zhang,6 Tingting Zhang,7 Chengwei Fu,1 Jie Yu,8 Weike Hu,9 Shengxian Peng9,* 

1Senior Department of Urology, the Third Medical Center of PLA General Hospital, Beijing, People’s Republic of China; 2Medical School of Chinese PLA, Beijing, People’s Republic of China; 3Medical Innovation Research Department, Chinese PLA General Hospital, Beijing, People’s Republic of China; 4Emergency and Critical Care Medical Center, Beijing Shijitan Hospital, Capital Medical University, Beijing, People’s Republic of China; 5Chongqing College of Electronic Engineering, Chongqing, People’s Republic of China; 6College of Medical Informatics, Chongqing Medical University, Chongqing, People’s Republic of China; 7Department of Endocrinology, Fifth Medical Center of Chinese PLA Hospital, Beijing, People’s Republic of China; 8Department of Medical Imaging, The Affiliated Taian City Central Hospital of Qingdao University, Taian, People’s Republic of China; 9Scientific Research Department, First People’s Hospital of Zigong City, Zigong, People’s Republic of China

*These authors contributed equally to this work

Correspondence: Shengxian Peng, Scientific Research Department, First People’s Hospital of Zigong City, No. 42, 1st Street, Shangyihao, ZiLiuJing District, ZiGong, SiChuan, 643000, People’s Republic of China, Email 13258280319@163.com

Background: : The traditional tool for predicting distant metastasis in renal cell carcinoma (RCC) is still insufficient. We aimed to establish an interpretable machine learning model for predicting distant metastasis in RCC patients.
Methods: We involved a population-based cohort of 121433 patients (mean age = 63 years; 63.58% men) diagnosed with RCC between 2004 and 2015 from the Surveillance, Epidemiology, and End Results (SEER) database. The lightGBM algorithm was used to develop prediction model and assessed by the area under the receiver-operating characteristic curve (AUC), accuracy, sensitivity, and specificity. The LightGBM model was then externally validated in 36395 RCC patients enrolled from the SEER database between 2016 and 2018. Shapley Additive exPlanations (SHAP) method was applied to provide insights into the model’s outcome or prediction.
Results: Of 121433 patients involved in the study cohort, 10730 (8.84%) had distant metastasis. The LightGBM model showed good performance in the internal validation set (AUC: 0.955, 95% CI: 0.951– 0.959) and temporal external validation sets (0.963, 95% CI: 0.959– 0.967; 0.961, 95% CI: 0.954– 0.966). Performance for the prediction model was also well performed in different sub-cohort stratified by age, gender, and ethnicity. The calibration curve indicated that the predicted values are highly consistent with the actual observed values. SHAP plots demonstrated that chemotherapy was the most vital variable for prediction of distant metastasis of RCC patients.
Conclusion: We developed an interpretable machine learning model that is capable of accurately predicting the risk of distant metastasis of RCC patients. The presented model could help identify high-risk patients who require additional treatment strategies and follow-up regimens.

Keywords: distant metastasis, machine learning, renal cell carcinoma, prediction, interpretable