Successive cold winters of severely low temperatures in recent years have had critical social and economic impacts on the mid-latitude continents in the Northern Hemisphere. Although these cold winters are thought to be partly driven by dramatic losses of Arctic sea ice, the mechanism that links sea ice loss to cold winters remains a subject of debate. Here, by conducting observational analyses and model experiments, we show how Arctic sea ice loss and cold winters in extra-polar regions are dynamically connected through the polar stratosphere. We find that decreased sea ice cover during early winter months (November-December), especially over the Barents-Kara (B-K) seas, enhance the upward propagation of planetary-scale waves with wavenumbers of 1 and 2, subsequently weakening the stratospheric polar vortex in mid-winter (January-February). The weakened polar vortex preferentially induces a negative phase of Arctic Oscillation (AO) at the surface, resulting in low temperatures in mid-latitudes.