Cancer patients’ participation in social, recreational, and civic activities is strongly associated with quality of life (QOL), but these activities are not well integrated into cancer survivorship research or interventions. Test the hypothesis that for long-term (≥ 5 years) survivors of rectal cancer, clinical factors (type of surgery and bowel function) are associated with long-term participation in activities and that participation in activities is associated with long-term QOL. Observational study with longitudinal and cross-sectional components. Participation in activities and QOL. Tumor registry records were used to identify patients and obtain clinical data; surveys assessed participation and QOL. Using general linear models, we analyzed participation in activities in relation to type of surgery and bowel function after adjustment for potential confounders. We analyzed overall QOL relative to participation in activities after adjustment. A total of 567 rectal cancer survivors completed a mailed questionnaire. Overall response rate was 61%. The type of operation (p < 0.0001), receipt of radiation therapy (p = 0.002), and bowel function (p < 0.0001) were associated with participation in activities. Participation in activities was the strongest predictor of QOL (p < 0.0001), explaining 20% of the variance (R2) in QOL, with all other variables together accounting for another 18% of the variance. The importance of participation in activities on rectal cancer survivors' QOL is underappreciated. We recommend revising QOL instruments used in cancer care and research to include questions about participation in activities. Interventions should address maintenance of preferred activities and adoption of new, fulfilling activities.