While glacier mass changes in the Himalaya since the year 2000 are relatively well investigated, there is still a lack of knowledge about the long-term changes and their climatic drivers. We use historical and recent remote sensing data to study glacier changes of the Lahaul-Spiti region in western Himalaya, India, over the last four decades (1971–2013). The glaciers were losing mass moderately between 1971 and 1999 (− 0.07 ± 0.1 m w.e. year−1) while the losses have increased significantly after 2000 (− 0.30 ± 0.1 m w.e. year−1). During both periods, the debris-covered glaciers and glaciers having pro-glacial lakes lost more mass than glaciers with little debris cover. Mass changes of Chhota Shigri, a benchmark glacier, closely matched the average of the overall study area. Analysis of gridded climate data covering the period 1948–2015 shows that the mean annual air temperature increased, especially since 1995. One dataset shows a significant increase in summer temperature after 2000 while others do not show any trend. The mean annual precipitation started decreasing around 1995 and reached a minimum around 2000, after which it increased again. One dataset shows a significant decrease in winter precipitation after 2000 while the others show no trend. The climate data indicate that the increase in mean annual temperature from 1995, combined with no significant trend/significant decrease of winter precipitation in the period after 2000, has probably resulted in accelerated mass loss of the glaciers.