When switching tasks, occasionally responding to bivalent stimuli (i.e., stimuli with relevant features for two different tasks) slows performance on subsequent univalent stimuli, even when they do not share relevant features with bivalent stimuli. This performance slowing is labelled the bivalency effect. Here, we investigated whether the bivalency effect results from an orienting response to the infrequent stimuli (i.e., the bivalent stimuli). To this end, we compared the impact of responding to infrequent univalent stimuli to the impact of responding to infrequent bivalent stimuli. For the latter, the results showed a performance slowing for all trials following bivalent stimuli. This indicates a long-lasting bivalency effect, replicating previous findings. For infrequent univalent stimuli, however, the results showed a smaller and shorter-lived performance slowing. These results demonstrate that the bivalency effect does not simply reflect an orienting response to infrequent stimuli. Rather it results from the conflict induced by bivalent stimuli, probably by episodic binding with the more demanding context created by them.