Job polarization is a widely documented phenomenon in developed countries since the 1980s: employment has been shifting from middle to low- and high-income workers, while average wage growth has been slower for middle-income workers than at both extremes. We document 1) that polarization has started as early as the 1950s in the US, and 2) that this process is closely linked to the shift from manufacturing to services. Based on these observations we propose a structural change driven explanation for polarization. Productivity growth through raising national income leads to a partial marketization of home production, and a disproportionate increase in the demand for high-end (luxury) services. To attract more workers into the low- and high-skilled services, the wages in these two sectors have to grow at a faster pace than in the middle.