The giant impact hypothesis for Moon formation successfully explains the dynamic properties of the Earth–Moon system but remains challenged by the similarity of isotopic fingerprints of the terrestrial and lunar mantles. Moreover, recent geochemical evidence suggests that the Earth's mantle preserves ancient (or "primordial") heterogeneity that pre-dates the Moon-forming giant impact. Using a new hydrodynamical method, we here show that Moon-forming giant impacts lead to a stratified starting condition for the evolution of the terrestrial mantle. The upper layer of the Earth is compositionally similar to the disk, out of which the Moon evolves, whereas the lower layer preserves proto-Earth characteristics. As long as this predicted compositional stratification can at least partially be preserved over the subsequent billions of years of Earth mantle convection, a compositional similarity between the Moon and the accessible Earth's mantle is a natural outcome of realistic and high-probability Moon-forming impact scenarios. The preservation of primordial heterogeneity in the modern Earth not only reconciles geochemical constraints but is also consistent with recent geophysical observations. Furthermore, for significant preservation of a proto-Earth reservoir, the bulk major-element composition of the Earth–Moon system may be systematically shifted toward chondritic values.