In the present study we investigated 90 musicians of both sexes who possess different degrees of absolute pitch (AP) using diffusion tensor imaging in association with a correlational approach and evaluated whether there are differences in perisylvian connections depending on the proficiency level of AP. We expected higher fractional anisotropy with increasing AP ability in the white matter underlying perisylvian brain areas. Results revealed a significant positive correlation between the proficiency level of AP and fractional anisotropy values in the left-sided white matter underlying the planum temporale. We interpret this result as an indicator of auditory processing differences between musicians possessing different degrees of AP, reflecting early auditory encoding and categorization processes. The present study provides further evidence for the substantial importance of the left-sided planum temporale for the ability of AP.