Genetic divergence by environment is a process whereby selection causes the formation of gene flow barriers between populations adapting to contrasting environments and is often considered to be the onset of speciation. Nevertheless, the extent to which genetic differentiation by environment on small spatial scales can be detected by means of neutral markers is still subject to debate. Previous research on the perennial herb Primula veris has shown that plants from grassland and forest habitats showed pronounced differences in phenology and flower morphology, suggesting limited gene flow between habitats. To test this hypothesis, we sampled 33 populations of P. veris consisting of forest and grassland patches and used clustering techniques and network analyses to identify sets of populations that are more connected to each other than to other sets of populations and estimated the timing of divergence. Our results showed that spatial genetic variation had a significantly modular structure and consisted of four well-defined modules that almost perfectly coincided with habitat features. Genetic divergence was estimated to have occurred about 114 generations ago, coinciding with historic major changes in the landscape. Overall, these results illustrate how populations adapting to different environments become structured genetically within landscapes on small spatial scales.