Physical activity offers many benefits to breast cancer survivors, yet research on physical activity during the immediate period following a breast cancer diagnosis is limited. In a prospective cohort study of 1,696 women diagnosed with invasive breast cancer in the Kaiser Permanente Northern California Medical Care Program from 2006-2009, we describe change in self-reported physical activity levels from around diagnosis to 6 months post-diagnosis and determine factors associated with change. Participants completed a comprehensive physical activity questionnaire at baseline (2 months post-diagnosis) and at follow-up (8 months post-diagnosis). Predictors of physical activity change were determined by multivariable linear regression. Reductions in all physical activity levels were observed (P < 0.0001); mean (SD) change (h/week) of moderate-vigorous physical activity (MVPA) was -1.28 (4.48) and sedentary behavior was -0.83 (6.95). In fully-adjusted models, overweight and obesity were associated with greater declines in MVPA of -1.58 h/week (SD = 0.92) and -1.29 h/week (SD = 0.93), respectively (P = 0.0079). Receipt of chemotherapy only was also associated with a greater decrease in MVPA (-2.12 h/week; SD = 0.92; P < 0.0001), specifically for recreational activities (-1.62 h/week; SD = 0.64; P = 0.0001). These data suggest challenges in maintaining physical activity levels during active treatment among women with breast cancer. Interventions to encourage physical activity in breast cancer survivors should be pursued.