Comparing patterns of diversity and divergence between populations at immune genes and neutral markers can give insights into the nature and geographic scale of parasite-mediated selection. To date, studies investigating such patterns of selection in vertebrates have primarily focused on the acquired branch of the immune system, whereas it remains largely unknown how parasite-mediated selection shapes innate immune genes both within and across vertebrate populations. Here, we present a study on the diversity and population differentiation at the innate immune gene Toll-like receptor 2 (TLR2) across nine populations of yellow-necked mice (Apodemus flavicollis) and bank voles (Myodes glareolus) in southern Sweden. In yellow-necked mice, TLR2 diversity was very low, as was TLR2 population differentiation compared to neutral loci. In contrast, several TLR2 haplotypes co-occurred at intermediate frequencies within and across bank vole populations, and pronounced isolation by distance between populations was observed. The diversity and differentiation at neutral loci was similar in the two species. These results indicate that parasite-mediated selection has been acting in dramatically different ways on a given immune gene in ecologically similar and sympatric species. Furthermore, the finding of TLR2 population differentiation at a small geographical scale in bank voles highlights that vertebrate innate immune defense may be evolutionarily more dynamic than has previously been appreciated.