Research on healthy aging in men has increasingly focused on age-related hormonal changes. Testosterone (T) decline is primarily investigated, while age-related changes in other sex steroids (dehydroepiandrosterone [DHEA], estradiol [E2], progesterone [P]) are mostly neglected. An integrated hormone parameter reflecting aging processes in men has yet to be identified. 271 self-reporting healthy men between 40 and 75 provided both psychometric data and saliva samples for hormone analysis. Correlation analysis between age and sex steroids revealed negative associations for the four sex steroids (T, DHEA, E2, and P). Principal component analysis including ten salivary analytes identified a principal component mainly unifying the variance of the four sex steroid hormones. Subsequent principal component analysis including the four sex steroids extracted the principal component of declining steroid hormones (DSH). Moderation analysis of the association between age and DSH revealed significant moderation effects for psychosocial factors such as depression, chronic stress and perceived general health. In conclusion, these results provide further evidence that sex steroids decline in aging men and that the integrated hormone parameter DSH and its rate of change can be used as biomarkers for healthy aging in men. Furthermore, the negative association of age and DSH is moderated by psychosocial factors.