Healthy aging is associated with weaker functional connectivity within resting state brain networks and stronger functional interaction between these networks. This phenomenon has been characterized as reduced functional segregation and has been investigated mainly in cross-sectional studies. Here, we used a longitudinal dataset which consisted of four occasions of resting state fMRI and psychometric cognitive ability data, collected from a sample of healthy older adults (baseline N = 232, age range: 64-87 y, age M = 70.8 y), to investigate the functional segregation of several well-defined resting state networks encompassing the whole brain. We characterized the ratio of within-network and between-network correlations via the well-established segregation index. Our findings showed a decrease over a 4-year interval in the functional segregation of the default mode, frontoparietal control and salience ventral attention networks. In contrast, we showed an increase in the segregation of the limbic network over the same interval. More importantly, the rate of change in functional segregation of the frontoparietal control network was associated with the rate of change in processing speed. These findings support the hypothesis of functional dedifferentiation in healthy aging as well as its role in cognitive function in elderly.