Transitivity is perhaps the most fundamental choice axiom and, therefore, almost all economic models assume that preferences are transitive. The empirical literature has regularly documented violations of transitivity, but these violations pose little problem as long as they are simply a result of somewhat-noisy decision making and not a reflection of the deterministic part of individuals’ preferences. However, what if transitivity violations reflect individuals’ genuinely nontransitive preferences? And how can we separate nontransitive preferences from noise-generated transitivity violations – a problem that so far appears unresolved? Here we tackle these fundamental questions on the basis of a newly developed, non-parametric method which uses response times and choice frequencies to distinguish genuine preferences from noise. We extend the method to allow for nontransitive choices, enabling us to identify the share of weak stochastic transitivity violations that is due to nontransitive preferences. By applying the method to two different datasets, we document that a sizeable proportion of transitivity violations reflect nontransitive preferences. Specifically, in the two datasets, 19% and 14% of all cycles of alternatives for which preferences are revealed involve genuinely nontransitive preferences. These violations cannot be accounted for by any noise or utility specification within the universe of random utility models.