Microglia are embryonically seeded macrophages that contribute to brain development, homeostasis, and pathologies. It is thus essential to decipher how microglial properties are temporally regulated by intrinsic and extrinsic factors, such as sexual identity and the microbiome. Here, we found that microglia undergo differentiation phases, discernable by transcriptomic signatures and chromatin accessibility landscapes, which can diverge in adult males and females. Remarkably, the absence of microbiome in germ-free mice had a time and sexually dimorphic impact both prenatally and postnatally: microglia were more profoundly perturbed in male embryos and female adults. Antibiotic treatment of adult mice triggered sexually biased microglial responses revealing both acute and long-term effects of microbiota depletion. Finally, human fetal microglia exhibited significant overlap with the murine transcriptomic signature. Our study shows that microglia respond to environmental challenges in a sex- and time-dependent manner from prenatal stages, with major implications for our understanding of microglial contributions to health and disease.