Background: The comprehensive geriatric assessment (CGA) is the gold standard in geriatric oncology to identify patients at high risk of adverse outcomes and optimize cancer and overall management. Many studies have demonstrated that CGA could modify oncologic treatment decision. However, there is little knowledge on which domains of the CGA are associated with this change. Moreover, the impact of frailty and physical performance on change in cancer treatment plan has been rarely assessed. Methods: This is a cross-sectional study of older patients with solid or hematologic cancer referred by oncologists for a geriatric evaluation before cancer treatment. A comprehensive geriatric assessment was performed by a multidisciplinary team to provide guidance for treatment decision. We performed a multivariate analysis to identify CGA domains associated with change in cancer treatment plan. Results: Four hundred eighteen patients, mean age 82.8 ± 5.5, were included between October 2011 and January 2016, and 384 of them were referred with an initial cancer treatment plan. This initial cancer treatment plan was changed in 64 patients (16.7%). In multivariate analysis, CGA domains associated with change in cancer treatment plan were cognitive impairment according to the MMSE score (p = 0.020), malnutrition according to the MNA score (p = 0.023), and low physical performance according to the Short Physical Performance Battery (p = 0.010). Conclusion: Cognition, malnutrition and low physical performance are significantly associated with change in cancer treatment plan in older adults with cancer. More studies are needed to evaluate their association with survival, treatment toxicity and quality of life. The role of physical performance should be specifically explored.