Background: The selection of women with hormone receptor-positive (HR+) early breast cancer (EBC) at high risk of relapse after five years (yrs.) of adjuvant aromatase inhibitors (AIs) is crucial, as the benefit of extending AIs is counterbalanced by toxicity. We developed a clinicopathological tool to estimate the residual risk of relapse after five years of adjuvant AIs. Methods: The Institut de Cancérologie de l’Ouest (ICO) database was used to determine a prognostic score of post-five-year AI relapse. Cox regression models estimated our score’s prognostic performance. Results: In total, 1105 women were included. Median follow-up was 44 months (IQR = 21–70) post-AI treatment. From the Cox models, we designed a dichotomous prognostic score including the number of macrometastases, age (>70 yrs. vs. ≤70 yrs.), tumor size (≥T2 vs. not), and mitotic activity (≥2 vs. not). Overall, 77.5% of patients were classified as being at low risk and 22.5% at high risk of late recurrence. Low-risk patients had a five- to ten-year local or distant recurrence risk of 7.6% (95% CI, 5.4% to 10.6%) as compared with 26.9% (95% CI, 19.9% to 35.7%) for the high-risk roup. Conclusion: In this study, we developed a simple tool to identify women at high risk of relapse despite completing five years of AIs.