The financial hardships and social isolation experienced during the COVID-19 pandemic have been found to adversely affect children’s developmental outcomes. While many studies thus far have focused on school-aged children and the pandemic-related impacts on their academic skills and behavior problems, relatively less is known about pandemic hardships and associations with children’s development during their early years. Using a racially and economically diverse sample, we examined whether hardships experienced during the pandemic were associated with children’s development with a particular focus on communication and socioemotional development. Participants from eight cohorts of the Environmental influences on Child Health Outcomes program provided data on pandemic-related financial and social hardships as well as child developmental outcomes. Financial hardship was defined as at least one parent experiencing job loss or change, and social hardship was defined as families’ quarantining from household members or extended family and friends. The development of children under 4 was assessed longitudinally, before and during the pandemic (N = 684), using the Ages and Stages Questionnaire (ASQ). The Generalized Estimating Equations, which accounted for within-child correlation, were used for analysis. Families from minority backgrounds and low socioeconomic status disproportionately experienced pandemic-related hardships. Male children had higher odds of experiencing negative changes in communication and personal social skills from pre- to during-pandemic visits (ORs ranged between 2.24 and 3.03 in analysis with binary ASQ outcomes and ranged from -0.34-0.36 in analyses with ASQ z-scores, ps = 0.000). Pandemic-related hardships in the social and financial areas did not explain within-individual changes in children’s developmental outcomes. Negative developmental changes from pre- to during-pandemic were found in boys, yet we did not find any associations between increased experience of pandemic-related hardships and children’s development. E how pandemic hardships affect development using a larger sample size and with longer follow-up is warranted.