Through this review of 25 clinical and experimental works on long-term musical memories in Alzheimer’s disease (AD) patients, we attempt to clarify the conceptual understanding of musical memories, identify their evolution across the stages of the pathology, and propose possible explanations concerning the neural and cognitive mechanisms that underpin the preservation and impairment of certain musical memories. After clarifying the different kind of musical memories, we investigated their alterations throughout AD’s progression from mild to severe stages. Both procedural and retrograde semantic memory seem relatively spared in AD, while episodic memory appears to be impaired early. Moreover, partial preservation of music encoding in AD can be revealed through paradigms that are especially designed for AD patients (relying on behavioral cues, using adapted settings, etc.). Although seldomly used, they would definitely help understanding the preserved capacities in every stage of AD. However, more research is needed to better understand this phenomenon and assess its specificity to music or other types of supports. These findings could lead to multiple applications in care settings and research designs, bringing more nuanced understanding of how long-term musical memory degrades throughout the course of AD, and should encourage us to prioritize patients’ preserved cognitive abilities in current AD recreational and care programs.