Synesthesia is defined as the involuntary and automatic perception of a stimulus in 2 or more sensory modalities (i.e., cross-modal linkage). Colored-hearing synesthetes experience colors when hearing tones or spoken utterances. Based on event-related potentials we employed electric brain tomography with high temporal resolution in colored-hearing synesthetes and nonsynesthetic controls during auditory verbal stimulation. The auditory-evoked potentials to words and letters were different between synesthetes and controls at the N1 and P2 components, showing longer latencies and lower amplitudes in synesthetes. The intracerebral sources of these components were estimated with low-resolution brain electromagnetic tomography and revealed stronger activation in synesthetes in left posterior inferior temporal regions, within the color area in the fusiform gyrus (V4), and in orbitofrontal brain regions (ventromedial and lateral). The differences occurred as early as 122 ms after stimulus onset. Our findings replicate and extend earlier reports with functional magnetic resonance imaging and positron emission tomography in colored-hearing synesthesia and contribute new information on the time course in synesthesia demonstrating the fast and possibly automatic processing of this unusual and remarkable phenomenon.