Few epidemiologic studies have investigated associations of air pollution with cognition in older adults, and none has specifically compared associations across particle sources. We investigated whether exposure to particulate air pollution, characterized by size and source, was associated with cognitive function and decline in cognitive function. We included participants of the Whitehall II cohort who were residents of greater London and who attended the medical examination in study wave 2007-2009 (n = 2867). Annual average concentrations of particulate matter (PM) (PM10 and PM2.5 from all sources and from traffic exhaust) were modeled at resolution of 20 × 20 m for 2003-2009. We investigated the relationship between exposure to particles and a cognitive battery composed of tests of reasoning, memory, and phonemic and semantic fluency. We also investigated exposure in relation to decline in these tests over 5 years. Mean age of participants was 66 (standard deviation = 6) years. All particle metrics were associated with lower scores in reasoning and memory measured in the 2007-2009 wave but not with lower verbal fluency. Higher PM2.5 of 1.1 μg/m (lag 4) was associated with a 0.03 (95% confidence interval = -0.06 to 0.002) 5-year decline in standardized memory score and a 0.04 (-0.07 to -0.01) decline when restricted to participants remaining in London between study waves. This study provides support for an association between particulate air pollution and some measures of cognitive function, as well as decline over time in cognition; however, it does not support the hypothesis that traffic-related particles are more strongly associated with cognitive function than particles from all sources.