Background. Evaluation of health-related quality of life (HRQoL) in patients with Alzheimer’s disease (AD) is necessary to ensure optimal management. Several scales for assessing HRQoL of patients with AD exist, in particular the Quality of Life in Alzheimer’s Disease (QoL-AD), which includes an evaluation by the caregiver of the patient’s HRQoL. The aim of this study was to identify factors associated with patient, caregiver and overall HRQoL as assessed by the QoL-AD. Methods. Cross-sectional multicenter study in subjects aged 65 years and older, with mild to moderate AD. HRQoL scores from the QoL-AD were recorded (3 scores, corresponding to patient, caregiver and overall), as well as sociodemographic variables for the patient and the caregiver, and data from the geriatric cognitive assessment (cognitive, psycho-behavioral, functional evaluations). Caregiver burden was evaluated using the Zarit caregiver burden scale. Factors associated with each QoL-AD score were identified by multivariate linear regression using t-tests and β estimations. Study was registered in Clinical Trial.gov (NCT02814773). Results. In total, 123 patients with AD were included. For the patient QoL-AD evaluation, depression was significantly associated with lower HRQoL (β = − 2.56 ± 1.28, p = 0.04), while polypharmacy (β = − 1.80 ± 0.99, p = 0.07) and anxiety (β = − 1.70 ± 1.01, p = 0.09) tended to be associated with lower HRQoL scores. In terms of caregiver evaluations, depression (β = − 3.46 ± 1.09, p = 0.002), polypharmacy (β = − 1.91 ± 0.92, p = 0.04) and the presence of caregiver burden (β = − 3.50 ± 0.91, p = 0.0002) were associated with lower HRQoL. For the overall evaluation, depression (β = − 3.26 ± 1.02, p = 0.002) and polypharmacy (β = − 1.85 ± 0.81, p = 0.03) were significantly related to lower HRQoL. Conclusions. Depression and polypharmacy were two factors influencing HRQoL in patients with AD, both by patient self-report and on the caregiver report. Thus, despite the discrepancies between HRQoL as assessed by patients with AD and HRQoL as assessed by their caregiver, the caregiver’s assessment may be used to guide patient management when the patient can no longer complete QoL evaluations. Moreover, the association between caregiver burden and the caregiver’s QoL-AD score underlines the need to take caregivers into consideration in the overall management of the AD patient.