In recent years, under the influence of the “global war for talent,” labor immigration policies in more developed economies (MDE) have been characterized by a dichotomization regarding foreign workers’ skills. While the immigration of highly skilled foreign workers is now being actively promoted, low-skilled immigration is being curbed by increasingly restrictive regulations. According to official immigration policy, Japan is an example par excellence of this pattern among MDE. However, in contradiction to its official immigration policy and like many other MDE, Japan has been experiencing a continuous inflow of non-highly skilled foreign workers and is structurally dependent on them today. This paper analyzes changing concepts of valuable foreign workers in Japan over the last three decades and their consequences. Who exactly a “valuable” foreign worker is is a highly contested question. It lies at the heart of Japan’s immigration policy debates and is embedded in changing ideational perceptions of immigration. Three reform periods in immigration policy and their long-term consequences are analyzed here: (1) the plan for increasing the number of foreign students as part of Japan’s internationalization in the 1980s; (2) the first reform debate in reaction to new irregular immigration around 1990; and (3) the second reform debate in view of Japan’s long-term demographic development in recent years.