Preferences over risky alternatives can be elicited by different methods, including direct pairwise choices and willingness-to-accept valuations. The results are frequently at odds, casting doubts on the foundations of economics. We develop a stochastic choice model predicting when inconsistencies across elicitation methods should occur, the type of anomalies to be expected, what determines their magnitude, and whether they uncover a bias or not. While some anomalies can be traced back to individual biases, other apparent anomalies can occur in the absence of any actual behavioral bias, as a consequence of regularities in stochastic choice, risk attitudes, and experimental design. The model delivers new predictions that are confirmed in five experiments on the classical preference reversal phenomenon. Our novel empirical approach relies on utilities estimated out of sample, which allow us to test the model and also show that the bias in willingness-to-accept valuations is limited to long shots.