Evaluating machine learning enhanced intelligent-optimization-engine (IOE) performance for ethos head-and-neck (HN) plan generation

Purpose: Varian Ethos utilizes novel intelligent-optimization-engine (IOE) designed to automate the planning. However, this introduced a black box approach to plan optimization and challenge for planners to improve plan quality. This study aims to evaluate machine-learning-guided initial reference plan generation approaches for head & neck (H&N) adaptive radiotherapy (ART). Methods: Twenty previously treated patients treated on C-arm/Ring-mounted were retroactively re-planned in the Ethos planning system using a fixed 18-beam intensity-modulated radiotherapy (IMRT) template. Clinical goals for IOE input were generated using (1) in-house deep-learning 3D-dose predictor (AI-Guided) (2) commercial knowledge-based planning (KBP) model with universal RTOG-based population criteria (KBP-RTOG) and (3) an RTOG-based constraint template only (RTOG) for in-depth analysis of IOE sensitivity. Similar training data was utilized for both models. Plans were optimized until their respective criteria were achieved or DVH-estimation band was satisfied. Plans were normalized such that the highest PTV dose level received 95% coverage. Target coverage, high-impact organs-at-risk (OAR) and plan deliverability was assessed in comparison to clinical (benchmark) plans. Statistical significance was evaluated using a paired two-tailed student t-test. Results: AI-guided plans were superior to both KBP-RTOG and RTOG-only plans with respect to clinical benchmark cases. Overall, OAR doses were comparable or improved with AI-guided plans versus benchmark, while they increased with KBP-RTOG and RTOG plans. However, all plans generally satisfied the RTOG criteria. Heterogeneity Index (HI) was on average <1.07 for all plans. Average modulation factor was 12.2 ± 1.9 (p = n.s), 13.1 ± 1.4 (p = <0.001), 11.5 ± 1.3 (p = n.s.) and 12.2 ± 1.9 for KBP-RTOG, AI-Guided, RTOG and benchmark plans, respectively. Conclusion: AI-guided plans were the highest quality. Both KBP-enabled and RTOG-only plans are feasible approaches as clinics adopt ART workflows. Similar to constrained optimization, the IOE is sensitive to clinical input goals and we recommend comparable input to an institution's planning directive dosimetric criteria.