We hypothesized that greater cardiorespiratory fitness is associated with lower odds of having unfavorable brain MRI findings. We studied 565 healthy, middle-aged, black and white men and women in the CARDIA (Coronary Artery Risk Development in Young Adults) Study. The fitness measure was symptom-limited maximal treadmill test duration (Maxdur); brain MRI was measured 5 years later. Brain MRI measures were analyzed as means and as proportions below the 15th percentile (above the 85th percentile for white matter abnormal tissue volume). Per 1-minute-higher Maxdur, the odds ratio for having less whole brain volume was 0.85 (p = 0.04) and for having low white matter integrity was 0.80 (p = 0.02), adjusted for age, race, sex, clinic, body mass index, smoking, alcohol, diet, physical activity, education, blood pressure, diabetes, total cholesterol, and lung function (plus intracranial volume for white matter integrity). No significant associations were observed between Maxdur and abnormal tissue volume or blood flow in white matter. Findings were similar for associations with continuous brain MRI measures. Greater physical fitness was associated with more brain volume and greater white matter integrity measured 5 years later in middle-aged adults.