Do voters polarize ideologically when radical views gain political legitimacy, or does the rise of radical voices merely reflect societal conflict? We argue that elite polarization as signaled by radical parties' first entrance into parliament leads to voter divergence. Immediately after the election, legitimization and backlash effects mean that voters on both ideological sides move toward the extremes. In the longer term, this polarization is solidified because of radical parties' parliamentary presence. A panel study of Dutch voters shows that the 2002 parliamentary entrance of a radical‐right party indeed led to immediate ideological polarization across the political spectrum. Estimating time‐series cross‐sectional models on Eurobarometer data from 17 countries (1973–2016) shows an additional long‐term impact of radical‐right party entry on polarization. The presence of radical voices on the right has polarizing effects, illustrating how such institutional recognition and legitimization can have a far‐reaching impact on society.