Both food insecurity (limited food access owing to cost) and living in areas with low physical access to nutritious foods are public health concerns, but their relative contribution to diabetes management is poorly understood. This was a prospective cohort study. A random sample of patients with diabetes in a primary care network completed food insecurity assessment in 2013. Low physical food access at the census tract level was defined as no supermarket within 1 mile in urban areas and 10 miles in rural areas. HbA1c measurements were obtained from electronic health records through November 2016. The relationship among food insecurity, low physical food access, and glycemic control (as defined by HbA1c) was analyzed using hierarchical linear mixed models. Three hundred and ninety-one participants were followed for a mean of 37 months. Twenty percent of respondents reported food insecurity, and 31% resided in an area of low physical food access. In adjusted models, food insecurity was associated with higher HbA1c (difference of 0.6% [6.6 mmol/mol], 95% CI 0.4-0.8 [4.4-8.7], P < 0.0001), which did not improve over time (P = 0.50). Living in an area with low physical food access was not associated with a difference in HbA1c (difference 0.2% [2.2 mmol/mol], 95% CI -0.2 to 0.5 [-2.2 to 5.6], P = 0.33) or with change over time (P = 0.07). Food insecurity is associated with higher HbA1c, but living in an area with low physical food access is not. Food insecurity screening and interventions may help improve glycemic control for vulnerable patients.