Reward magnitude is a central concept in most theories of preferential decision making and learning. However, it is unknown whether variable rewards also influence cognitive processes when learning how to make accurate decisions (e.g., sorting healthy and unhealthy food differing in appeal). To test this, we conducted 3 studies. Participants learned to classify objects with 3 feature dimensions into two categories before solving a transfer task with novel objects. During learning, we rewarded all correct decisions, but specific category exemplars yielded a 10 times higher reward (high vs. low). Counterintuitively, categorization performance did not increase for high-reward stimuli, compared with an equal-reward baseline condition. Instead, performance decreased reliably for low-reward stimuli. To analyze the influence of reward magnitude on category generalization, we implemented an exemplar-categorization model and a cue-weighting model using a Bayesian modeling approach. We tested whether reward magnitude affects (a) the availability of exemplars in memory, (b) their psychological similarity to the stimulus, or (c) attention to stimulus features. In all studies, the evidence favored the hypothesis that reward magnitude affects the similarity gradients of high-reward exemplars compared with the equal-reward baseline. The results from additional reward-judgment tasks (Studies 2 and 3) strongly suggest that the cognitive processes of reward-value generalization parallel those of category generalization. Overall, the studies provide insights highlighting the need for integrating reward- and category-learning theories.