Nestled into a small valley between the mansions of Dayton's Bluff and St. Paul proper, Swede Hollow was a bustling community tucked away from the prying eyes of the city above. It lacked more than it offered; houses had no plumbing, electricity, or yards, and there were no roads or businesses. In spite of this, it provided a home to the poorest immigrants in St. Paul for nearly a century.

Minnesota Territory experienced a boom period starting in 1855. Industry flourished region-wide and companies amassed incredible wealth. The Financial Panic of 1857 brought the good times to a halt and interrupted the growth of the fledgling state.

The K.J. Taralseth Company building is a physical reminder of the early commercial development of Warren. After moving from a brick store that was destroyed by fire in 1910, Ralph Taralseth built a new store that reflected the company's success. The new building carried a mixed product line for which the company became known. It also provided space for the professional services and fraternal organizations forming in and around Warren.