Introduction Regular moderate‐to‐vigorous physical activity (MVPA) is associated with improved mental health, but the evidence for the effect of reducing sedentary behavior (SB) or increasing light PA (LPA) in older adults is lacking. Using isotemporal substitution (IS) models, the aim of this paper was to investigate the effect of substituting SB with LPA or MVPA on associations with mental health in older adults. Methods Data from 1360 older adults (mean age 75.18 years) in four countries were utilized. PA and SB was measured using ActiGraph wGT3X‐BT + accelerometers worn for 7 days. Self‐rated mental health was measured using the Hospital and Anxiety Depression Scale (HADS). IS models estimated cross‐sectional associations when 30 minutes of one behavior was substituted with another. Models were adjusted for age, sex, marital status, and educational attainment. Results Substituting 30 minutes of SB with LPA (β −.37; 95% CI −0.42, −0.32) or MVPA (β −.14; 95% CI −0.21, −0.07) and substituting LPA with MVPA (β −.11; 95% CI −0.18, −0.04) were associated with improvements in anxiety. However, substituting 30 minutes of SB with LPA (β .55; 95% CI 0.49, 0.62) was associated with increased depression. Conclusion Replacing 30 minutes of SB with LPA or MVPA was associated with improved anxiety symptoms in older adults. Greater benefits were observed when shifting SB and LPA to MVPA.