Space, numbers and time share similar processing mechanisms mediated by parietal cortex. In parallel to the spatial representation of numbers along a horizontal line, temporal information is mapped on a horizontal axis with short intervals (and the past) represented to the left of long intervals (and the future). Little is known about the representation of time in the presence of visuo-spatial deficits. We here report two experiments on the comparative judgment of time. Experiment 1 required patients with left-sided neglect to indicate which of two consecutively presented silent intervals was longer. Their judgments were better if the first interval was longer and they judged the first interval longer on trials in which the two intervals were equally long. These results were not present in right-hemispheric damaged patients without neglect and healthy controls. They are in line with a previously reported finding in a single patient with neglect, but not readily compatible with findings of neglect patients' comparative length judgments. In Experiment 2, healthy participants' performance on an identical task improved for trials with a first-longer interval after caloric vestibular stimulation (CVS) of the right ear with warm water.