Background: Olive oil is a major component of the Mediterranean diet suggested to be beneficial to counteract Alzheimer's disease. Aim of the Study: Our objective was to examine the association between olive oil use, cognitive deficit and cognitive decline in a large elderly population. Methods: We followed 6,947 subjects with a brief baseline food frequency questionnaire and repeated cognitive tests. Olive oil intake was categorized as none (22.7%), moderate (use for cooking or dressing, 39.9%) and intensive (use for both cooking and dressing, 37.4%). Associations between olive oil and cognitive outcomes were examined taking into account socio-economic factors, health behaviors, health measures and other dietary intakes. Results: Participants with moderate or intensive use of olive oil compared to those who never used olive oil showed lower odds of cognitive deficit for verbal fluency and visual memory. For cognitive decline during the 4-year follow-up, the association with intensive use was significant for visual memory (adjusted OR = 0.83, 95% CI: 0.69-0.99) but not for verbal fluency (OR = 0.85, 95% CI: 0.70-1.03) in multivariate analysis. Conclusions: This olive oil-cognition association needs to be confirmed by further studies. However, our findings already shed light on the potential importance of olive oil in the Mediterranean diet and on its beneficial effects on health.