Human pancreatic ductal adenocarcinoma (PDAC) responds poorly to immune checkpoint inhibitor (ICPi). While the mechanism is not completely clear, it has been recognized that tumor microenvironment (TME) plays key roles. We investigated if targeting CD47 with a monoclonal antibody could enhance the response of PDAC to ICPi by altering the TME. Using immunohistochemistry, we examined tumor-infiltrating CD68+ pan-macrophages (CD68+ M) and CD163+ M2 macrophages (CD163+ M2) and tumor expression of CD47 and PD-L1 proteins in 106 cases of PDAC. The efficacy of CD47 blockade was examined in xenograft models. CD45+ immune cells from syngeneic tumor models were subjected to single-cell RNA-sequencing (scRNA-seq) by using the 10x Genomics pipeline. We found that CD47 expression correlated with the level of CD68+ M but not CD163+ M2. High levels of tumor-infiltrating CD68+ M, CD163+ M2, and CD47 expression were significantly associated with worse survival. CD47high/CD68+ Mhigh and CD47high/CD163+ M2high correlated significantly with shorter survival, whereas CD47low/CD68+ Mlow and CD47low/CD163+ M2low correlated with longer survival. Intriguingly, CD47 blockade decreased the tumor burden in the Panc02 but not in the MPC-83 syngeneic mouse model. Using scRNA-seq, we showed that anti-CD47 treatment significantly remodeled the intratumoral lymphocyte and macrophage compartments in Panc02 tumor-bearing mice by increasing the pro-inflammatory macrophages that exhibit anti-tumor function, while reducing the anti-inflammatory macrophages. Moreover, CD47 blockade not only increased the number of intratumoral CD8+ T cells, but also remodeled the T cell cluster toward a more activated one. Further, combination therapy targeting both CD47 and PD-L1 resulted in synergistic inhibition of PDAC growth in the MPC-83 but not in Panc02 model. MPC-83 but not Panc02 mice treated with both anti-CD47 and anti-PD-L1 showed increased number of PD-1+CD8+ T cells and enhanced expression of key immune activating genes. Our data indicate that CD47 targeting induces compartmental remodeling of tumor-infiltrating immune cells of the TME in PDAC. Different PDAC mouse models exhibited differential response to the anti-CD47 and anti-PD-L1 blockade due to the differential effect of this combination treatment on the infiltrating immune cells and key immune activating genes in the TME established by the different PDAC cell lines.